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ABSTRACT 

This  thesis  examines  the  initial  decision  process  of  organizing  to  convert  a  course 
module  from  the  Navy  Medical  Department  Executive  Management  Education.  The 
objective  is  to  track  and  model  the  process  used  to  integrate  Network-Based  Instruction 
technology  into  an  existing  traditional  classroom  course  of  instruction.  This  research 
include  survey  of  Network-Based  Instruction  literature,  and  an  assessment  of  the  decision 
process  involved.  The  goal  is  to  capture  the  context  of  the  key  decisions  made  during  the 
preliminary  stage  of  an  actual  conversion  project,  analyze  the  effectiveness  of  the  approach 
and  if  possible  generate  a  model  for  future  efforts. 
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I.  INTRODUCTION 

This  research  will  examine  the  decision  process  behind  the  conversion  of  a  course 
module  from  the  Navy  Medical  Department  Executive  Management  Education  (EME) 
curriculum  to  Network  Base  Instruction  (NBI).  The  objective  is  to  track  and  model  the 
process  used  to  integrate  NBI  technology  into  an  existing  traditional  classroom  course  of 
instruction.  Research  will  include  a  survey  of  literature  on  NBI  technology,  and  an 
assessment  of  the  decision  process  involved.  The  goal  is  to  capture  the  context  of  the  key 
decisions  made  during  the  preliminary  stage  of  an  actual  conversion  project,  analyze  the 
effectiveness  of  the  approach,  and,  if  possible,  generate  a  model  for  future  conversion 
efforts. 

A.  RESEARCH  QUESTIONS 

Primary  Research  Question: 

1 .    What  process  was  used  to  transform  the  current  EME  module  to  a  NBI  version? 
Subsidiary  Research  Questions: 

1 .  Who  were  the  stakeholders? 

2.  What  was  the  perceived  value  of  the  conversion  to  NBI? 

3.  What  were  the  significant  resources  and  constraints  on  this  conversion  project? 

B.  SCOPE 

The  scope  will  include:  (1)  tracking  the  decision  process  in  the  preliminary  stage  of 
converting  an  existing  EME  course  module  to  NBI,  (2)  evaluating  the  process  of  converting 
current  EME  course  module  to  NBI,  and  (3)  reviewing  costs  associated  with  changing  the 
process.  The  thesis  will  conclude  with  a  proposed  template  for  the  process  of  converting  a 
course  module  to  NBI. 


C.  METHODOLOGY 

The  methodology  used  in  this  thesis  research  consists  of  the  following  steps:  (1) 
Conduct  a  literature  search  of  applicable  Department  of  Defense  (DoD)  instructions, 
notices,  studies,  command  policies,  books,  magazine  articles,  and  other  information  relevant 
to  EME  and  NBI;  (2)  Conduct  interviews  with  key  players  at  the  Bureau  of  Medicine  and 
Surgery  (BUMED),  Naval  Postgraduate  School  (NPS),  and  vendors;  (3)  Track  and  model 
the  process  used  to  integrate  NBI  technology  into  an  existing  EME  course  module;  (4) 
Prepare  an  assessment  of  the  decision  process  used  to  integrate  NBI  technology  into  EME 
course  module,  (5)  Identify  potential  cost  savings  from  changes  to  the  existing  process. 

D.  BENEFITS  OF  THE  STUDY 

This  study  will  provide  the  information  required  to  better  manage  the  conversion 
process  of  an  existing  EME  curriculum  to  NBI. 

E.  CHAPTER  OUTLINE 

Chapter  I  discusses  the  purpose,  scope,  methodology,  and  benefits  of  the  study. 
Chapter  II  provides  the  background  about  the  Congressional  mandate,  initial  DoD  and  Navy 
administrative  health  care  assessment,  the  NPS  Executive  Management  Education  Program, 
and  EME  Stakeholders.  In  addition,  the  chapter  provides  a  literature  review  of  Network 
Based  Instruction.  Chapter  III  describes  the  decision  processes  used  in  the  conversion  of  an 
EME  course  module  to  NBI.  Chapter  IV  presents  an  assessment  of  the  decision  process 
utilized.  Chapter  V  provides  conclusions  and  recommendations  based  on  the  analysis 
presented  in  Chapter  IV. 


II.  BACKGROUND 

A.         CONGRESSIONAL  MANDATE 

The  U.S.  Congress,  in  1992  and  1996,  issued  specific  directions  to  the  Department 
of  Defense  (DoD)  concerning  the  preparation  of  officers  to  command  military  medical 
treatment  facilities  (MTF)  and  to  serve  as  TRICARE  lead  agents  or  key  members  of  the 
lead  agent  staff.  This  initiative  continues  today  to  meet  its  obligations  in  preparing  those 
officers  for  their  duties. 

To  direct  an  MTF  in  the  fulfillment  of  its  mission,  the  effective  commander  must 
possess  a  mixture  of  specific  skills,  knowledge,  and  experience.  This  mix  produces 
competencies  that  vary  with  the  education  and  life  experiences  of  the  individual.  The  need 
for  specific  competencies  is  particularly  evident  for  MTF  commanders,  lead  agents,  and 
their  key  staff  members,  who  must  have  both  general  management  skills  and  a  thorough 
understanding  of  the  health  care  delivery  process.  Commanders  must  successfully  manage 
the  various  types  of  complex  organizations  that  exist  in  the  DoD  health  care  environment. 

The  commander  of  an  MTF  and  the  lead  agent  of  a  TRICARE  region  are  in  unique 
positions  when  compared  to  their  civilian  peers.  In  addition  to  having  health  care 
administration  skills  similar  to  those  in  the  civilian  sector,  they  must  also  fulfill  military 
requirements.  Military  requirements  include  readiness  preparation,  deployment  skills, 
support  for  war  fighting,  direction  of  joint  military  and  civilian  health  care  networks,  and  a 
number  of  other  requirements.  Therefore,  the  necessary  skills  and  knowledge  of  an  MTF 
commander  or  lead  agent  not  only  must  match,  but  also  must  exceed,  those  of  their  civilian 
counterparts. 

The  specific  needs  of  MTF  commanders  were  emphasized  by  the  DoD 
Appropriations  Act  of  1992,  Section  8096,  which  stated:  "None  of  the  funds  appropriated  in 
this  Act  may  be  used  to  fill  the  commander's  position  at  any  military  treatment  facility  with 
a  health  care  professional  unless  the  prospective  candidate  can  demonstrate  professional 


administrative  skills."    Similarly,  the  1996  Defense  Authorization  Act  (Public  Law  104- 

106,   Title   VII,   Subtitle   B,   Section   715),   which  focuses   on  training   in  health  care 

management  and  administration  for  TRICARE  lead  agents  states: 

(a)PROVISIONS  OF  TRAINING.  Not  later  than  six  months  after  the  date 
of  the  enactment  of  this  Act,  the  Secretary  of  Defense  shall  implement  a 
professional  educational  program  to  provide  appropriate  training  in  health 
care  management  and  administration. 

( 1 )  to  each  commander  of  a  military  medical  treatment  facility  of  the  DoD 
who  is  selected  to  serve  as  a  lead  agent  to  coordinate  the  delivery  of  health 
care  by  military  and  civilian  providers  under  the  TRICARE  program;  and 

(2)  to  appropriate  members  of  the  support  staff  of  the  treatment  facility  who 
will  be  responsible  for  daily  operations  of  the  TRICARE  program. 

(b)REPORT  ON  IMPLEMENTATION.  Not  later  than  six  months  after  the 
date  of  the  enactment  of  this  Act,  the  Secretary  of  Defense  shall  submit  to 
Congress  a  report  describing  the  professional  educational  program 
implemented  pursuant  to  this  section.  [Ref.  1] 


B.         INITIAL  DOD  ASSESSMENTS 

In  December  1991,  the  Assistant  Secretary  of  Defense,  Health  Affairs,  in  response 
to  the  1992  Appropriations  Act,  convened  a  working  group  representing  the  Army,  Air 
Force,  Navy,  and  Coast  Guard  to  review  and  identify  professional  administrative  skills 
necessary  for  command  of  an  MTF.  The  group  reviewed  several  areas,  including  the 
operation  of  civilian  and  military  hospitals,  the  responsibilities  of  senior  leaders  in  both 
settings,  service  programs  in  place  to  prepare  individuals  for  command,  and  civilian 
standards  for  successful  leadership  in  a  health  care  organization.  They  identified  the  skills 
and  experience  necessary  for  successful  job  performance,  discussed  alternatives,  and 
determined  documentation  requirements.  Additionally,  they  identified  and  described  34 
competencies  needed  to  command  an  MTF,  along  with  suggested  procedures  for  validating 
candidates'  possession  of  these  competencies  during  the  command  selection  process.  [Ref. 

1] 


C.  THE  NAVY  INITIATIVE 

The  Navy's  Bureau  of  Medicine  and  Surgery  (BUMED),  in  partnership  with  the 
Naval  Postgraduate  School,  has  designed  a  program  based  on  the  competencies  identified  in 
the  studies  cited.  The  program.  Executive  Management  Education  (EME),  is  designed  to 
meet  the  requirements  of  the  Appropriation  Act.  The  EME  Program  is  being  offered  at 
different  MTF  sites  and  executive  sessions  at  NPS,  Monterey,  California. 

During  extensive  interviews  and  surveys  at  the  Navy's  major  MTFs,  senior  Navy 
health  care  executives  told  NPS  faculty  what  was  important  to  manage  their  commands  and 
where  they  saw  themselves  in  terms  of  competency  levels.  This  information  provided  the 
bases  for  designing  the  49  education  modules. 

The  modules  are  taught  by  full-time  NPS  graduate  faculty  at  the  Monterey  campus 
or  at  MTF  sites.  The  subject  matter  within  the  modules  was  created  as  a  result  of  the  need 
assessment.  These  modules  are  shorter  than  the  courses  in  the  health  care  management 
programs  of  several  universities,  which  require  one  or  two  years  of  residency.  Additional 
instructional  areas  that  relate  exclusively  to  managing  Navy  MTFs  have  been  integrated 
into  the  program.  [Ref.  2] 

D.  THE  EXECUTIVE  MANAGEMENT  EDUCATION 

The  EME  Program  is  designed  to  prepare  medical  department  officers  for  executive 
medicine  positions.  The  objectives  of  this  educational  program  are  1)  to  refine  a  manager's 
administrative  viewpoint,  which  considers  the  effects  of  any  one  decision  on  the 
organization,  2)  to  provide  a  thorough  understanding  of  the  precise  tools  associated  with 
operations  analysis  and  economic  consequences  of  any  decision,  and  3)  to  illuminate  the 
meaning  of  working  in  managed  care  and  lead  agency  environments.  These  objectives  aim 
to  provide  an  executive  with  increased  confidence  in  coping  with  both  present  and  future 
military  health  care  management. 


E.         STAKEHOLDERS 

A  stakeholder  is  any  group  that  can  affect  or  is  affected  by  the  achievement  of  an 
organization's  objectives.  [Ref.  3]  To  be  a  stakeholder  means  to  have  an  interest  in  the 
process  or  outcome  of  the  issue.  Some  stakeholders  may  want  things  to  stay  the  same; 
some  may  favor  small  changes  of  various  sorts;  some  may  want  radical  change.  There  may 
be  stakeholders  who  are  not  involved  in  any  controversy  or  process,  yet  have  an  interest  in 
the  issue.  Active  stakeholders  are  organizations  or  groups  or  occasional  individuals  who 
seek  to  affect  the  current  initiative  because  of  some  interest.  Some  of  them  may  be  public 
spirited  or  acting  on  behalf  of  others. 

The  EME  initiative  has  six  stakeholders,  not  all  of  whom  are  equally  affected  by  or 
have  a  significant  effect  on  the  initiative.  A  stakeholder  map  is  illustrated  in  Figure  1 . 

1 .  Commanding  Officers  and  senior  military  personnel  are  the  most  significantly 
affected  by  this  initiative.  They  are  required  to  complete  the  EME  curriculum  in 
order  to  acquire  the  necessary  proficiency  and  demonstrate  professional 
administrative  skills. 

2.  Congress  is  also  an  important  stakeholder  because  they  are  the  primary  decision 
maker  who  issued  a  directive  to  the  DoD  concerning  the  required  skills, 
knowledge,  and  experiences  an  MTF  commander  must  possess. 

3.  Clearly,  DoD,  including  the  individual  military  services,  has  a  major  stake  in 
this  EME  initiative.  The  DoD's  position  is  that  the  training  of  senior  medical 
personnel  is  vital  for  health  care  executives  to  achieve  the  necessary  skills. 

4.  Professors  who  are  experts  in  their  respective  fields  comprise  another  group  that 
can  exert  influence.  The  obvious  impact  of  this  initiative  is  that,  as  more 
medical  officers  go  through  this  program,  more  professors  are  needed  to  teach 
the  required  skills. 

5.  BUMED  has  a  direct  impact  on  the  EME  initiative.  They  are  involved  in  the 
design  and  funding  of  the  EME  program  based  on  the  competencies  identified. 

6.  NPS  Institute  for  Defense  Education  and  Analysis  (IDEA)  is  charge  with  the 
development,  delivery  and  management  of  EME  programs. 


Commanding  Officer  & 
Senior  Military  Personnel 


NPS  Institute  for  Defense  Education 
and  Analysis  (IDEA) 


Professors 


Congress 


Bureau  of  Medicine 
and  Surgery 


Department  of 
Defense 


Figure  1.  A  Stakeholder  Map 
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DISTANCE  LEARNING 


In  today's  world,  continuous  learning  has  become  essential  for  effective  military 
and  health  care  leaders.  Technology  is  quickly  strengthening  our  ability  to  respond  to  that 
need,  with  distance  education  emerging  as  a  timely  and  potentially  cost-effective  way  to 
provide  educational  opportunities. 

Non-traditional  education  is  increasingly  identified  with  distance  education  or 
education  taking  place  beyond  the  physical  spaces  (campuses)  traditionally  set  aside  for 
such  learning.  At  its  most  basic  level,  distance  education  takes  place  when  instructor  and 
srudent(s)  are  separated  by  physical  distance,  and  technology  is  used  to  bridge  the  gap. 
These  types  of  programs  can  provide   various  educational   opportunities  to   students. 


including  executives  in  both  the  public  and  private  sectors,  particularly  those  disadvantaged 
by  limited  time,  travel  budgets,  or  physical  disability.  [Ref.  4] 

Distance  education  has  always  benefited  from  the  development  of  communications. 
There  are  a  number  of  current  distance  learning  technologies  available,  such  as  microwave 
broadcasts  of  both  audio  and  video,  videoteleconferencing,  and  Network  Based  Instruction. 

The  microwave  broadcast  classroom  normally  contains  a  single  camera,  which 
concentrates  on  the  instructor,  on  an  experiment,  or  on  the  blackboards  where  notes  are 
being  written. 

Videoteleconferencing  is  an  interactive  distance  learning  setup  using  the  t-1  phone 
line,  which  provides  two-way  audio  and  video  between  the  broadcast  and  remote 
classrooms.  Major  universities  and  school  districts  throughout  the  country  are  now  wired 
for  the  Integrated  Services  Digital  Network  (ISDN),  which  uses  compression  technology  to 
transmit  voice,  video,  and  data  simultaneously  over  standard  telephone  lines.  [Ref.  5] 

The  most  recent  distance  learning  medium  of  learning  and  teaching  is  Network 
Based  Instruction  (NBI).  This  innovative  approach  for  delivering  instruction  to  remote 
audience  is  accomplished  by  using  the  Web.  Public  and  private  organizations  like  DoD, 
colleges  and  universities;  corporations  like  IBM,  Microsoft,  and  others  are  investing 
resources  in  developing  these  new  training  modalities.  They  expect  an  increase  in  Web- 
based  training  within  the  next  few  years.  [Ref.  5]  Organizations,  NPS  among  them,  whose 
customers  are  limited  by  cost  and  time  constraints  will  require  the  flexibility  that  Web- 
based  training  can  provide.  NPS  launched  three  Web-based  educational  programs  in  1 997 
and  plans  additional  online  programming  for  EME  this  year. 

G.         NETWORK-BASED  INSTRUCTION 

Network-Based  Instruction  is  defined  as  a  hypermedia-based  instructional  program, 
which  utilizes  the  attributes  and  resources  of  the  World  Wide  Web  to  create  a  meaningful 
environment  where  learning  is  fostered  and  supported.  [Ref.  6] 


1.  Origins 

The  Internet  began  as  a  project  by  the  Defense  Advance  Research  Project  Agency 
(DARPA),  whose  goal  was  to  develop  a  highly  decentralized  network  that  could  both  route 
information  and  survive  a  nuclear  war  or  natural  disasters.  The  resulting  network  was 
originally  used  for  sharing  information  between  research  laboratories  and  researchers  at 
educational  institutions.  Eventually,  more  and  more  students  began  to  use  the  network,  and 
when  companies  joined  in  as  well,  complete  with  advertising,  the  Internet  was  born. 

One  of  the  earliest  uses  of  the  Internet  was  for  electronic  mail.  Long  before  the 
World  Wide  Web  (WWW)  and  Web  pages,  the  net  was  used  for  conducting  searches  and 
finding  and  downloading  files  and  programs  (FTP).  Another  early  use  was  the  Usenet,  used 
by  news  groups,  which  was  developed  in  part  to  deal  with  the  heavy  volume  of  e-mail 
generated  by  list  servers.  The  hypertext  transfer  protocol  (http)  was  developed  in  the 
summer  of  1991  at  European  Particle  Physics  Laboratory  in  Geneva,  Switzerland  as  a  way 
to  share  research  papers,  and  the  first  Web  pages  were  created.  [Ref.  7] 

2.  Why  Network-Based  Instruction? 

The  use  of  the  WWW  is  growing  exponentially.  Estimates  range  from  an  increase 
of  six  percent  to  one  of  20  percent  per  month.  In  January  of  1991,  there  were  365,000 
hosts.  In  January  of  1996,  there  were  9,472,000  hosts.  [Ref.  6]  In  addition  to  user  traffic, 
the  creation  of  the  Web  servers,  home  pages,  and  other  digital  resources  on  the  Web  is 
similarly  expanding.    This  increase  represents  a  tremendous  potential  for  educators.    [Ref. 

8] 

The  WWW  and  the  associated  use  of  hypertext  approaches  have  created  new 
opportunities.  Over  the  last  year,  a  number  of  universities  experimented  with  and  started  to 
use  the  Internet  to  support  teaching  and  administration.  Harvard  Business  School  is 
offering  executive  education  via  the  Web.  The  use  of  Web  to  support  part-time  MBA  was 
seen  as  high  potential,  as  evidenced  by  the  University  of  Virginia  offering  Executive  MBA 
for  part-time  students.  [Ref.  9] 


One  success  story  in  providing  higher  education  through  NBI  is  the  University  of 
Phoenix,  a  no  frills  university,  with  no  laboratories,  gymnasiums,  and  dormitories.  They 
have  58  campuses  in  12  states,  Puerto  Rico,  and  dozens  of  foreign  countries.  As  of 
December  1997,  they  have  an  enrollment  of  42,000,  and  are  still  growing  at  20  percent  a 
year.  This  institution  provides  continuing  education,  bachelor  and  master  degree  programs 
mostly  in  business  and  information  technology  to  adults  who  work  full-time  and  attend 
classes  only  at  night  or  online.  Their  online  students  can  complete  their  university  studies 
from  virtually  anywhere  in  the  world.  [Ref.  10] 

NBI  is  particularly  applicable  for  MBA  and  executive  education  courses,  just  like 
the  EME  courses  for  senior  DoD  medical  personnel.  Students  can  enroll  and  participate  in 
these  on-line  courses  by  using  the  Intranet  applications.  Intranets  are  internal  Web  systems 
that  work  in  conjunction  with  the  Internet  and  WWW.  They  enable  students  or  employees 
to  access  school  or  company  information  with  appropriate  passwords. 

The  military  has  adapted  to  changing  technology  in  providing  training  and 
education  via  the  Web.  One  example  is  the  Defense  Systems  Agency  Information  Training 
Facility,  located  in  Falls  Church,  Virginia,  which  has  developed  and  distributed  Information 
Security  Training  Courses  to  various  Information  Security  agencies  via  the  Web.  [Ref.  11] 

The  Services  accomplish  most  of  their  training  of  individuals  in  residential  school 
settings.  For  several  thousand  courses,  the  Services  bear  the  travel  and  temporary  duty 
(TDY)  costs  needed  to  support  students  in  centralized  training  facilities.  This  practice 
affects  service  operations,  unit  performance,  and  quality  of  life  for  individuals  whose  work 
schedule  and  family  responsibilities  are  interrupted  by  travel  to  distant  training  locations. 
These  factors  have  motivated  the  Services  to  look  for  alternatives  to  centralized  training 
facilities.  Currently,  the  need  to  identify  acceptable  alternatives  like  NBI  is  increasing 
because:  (1)  budget  pressures  are  reducing  funds  to  operate  centralized  schools;  and  (2)  with 
modernization  and  technology  enhancements  that  increase  the  complexity  of  military 
equipment  and  support  systems,  training  demands  are  increasing.  [Ref.  1 2] 
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The  Naval  Postgraduate  School,  through  IDEA  as  the  technology  leader  within  DoD 
for  adapting  new  technology  to  education,  started  the  process  of  converting  EME  modules 
to  NBI.  This  initiative  will  provide  opportunity  for  anytime,  anywhere  EME  education  on 
demand  in  both  synchronous  and  asynchronous  environments.  Synchronous  instruction 
requires  the  simultaneous  participation  of  all  students  and  instructors,  and  the  interaction  is 
done  in  real  time.  Asynchronous  instruction  does  not  require  the  simultaneous  participation 
of  all  students  and  instructors.  They  do  not  need  to  gather  together  in  the  same  location  at 
the  same  time. 

As  an  instructional  technology,  the  Web  has  the  ability  to  carry  a  variety  of  media 
from  virtually  anywhere,  well  beyond  the  traditional  means.  With  access  to  course 
information  and  instruction  on  a  flexible  schedule  through  telecommunications,  students 
can  work  at  their  own  pace  and  to  have  more  time  to  compose  responses.  They  can  learn 
the  coursework  at  home  or  work,  wherever  and  whenever  Internet  access  is  available.  [Ref. 

13] 

3.  Educational  Uses  of  the  Internet 

As  indicated  earlier,  NBI  is  a  form  of  distance  learning  in  which  course  modules  are 
posted  on  the  WWW.  Individuals  with  access  can  log  on  and  connect  to  the  courses  at  their 
convenience.  Presentations  on  the  Internet  allow  for  multiple  media  types,  including  text, 
sound,  graphics  and  videos.  Finding  information  on  the  Internet  was  made  easy  through  the 
use  of  hypertext  and  hypermedia,  providing  links  to  a  new  page  with  related  information. 

The  Internet  itself  is  not  interactive.  It  can  be  used  in  many  ways  that  vary  from 
passive  reading,  like  a  library  book,  to  user  driven  simulations  that  change  direction  based 
on  choices  made  by  the  user,  like  the  video  game  model.  Depending  on  the  instructor's 
time,  resources,  and  imagination,  the  Web  and  the  rest  of  the  Internet  can  be  used  in 
minimally,  moderately,  or  highly  interactive  ways.  Interactivity  is  a  medium's  ability  for 
senders  and  receivers  to  notice  and  respond  to  each  other's  communication  cues. 
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a.  Minimally  Interactive  Use  of  the  Internet 

Minimally  interactive  use  of  Internet  is  the  lowest  level  of  interactivity. 
There  is  a  limited  or  no  interaction  between  the  student,  instructor,  or  expert  at  the  distant 
locations.  Listed  below  are  several  examples  of  minimally  interactive  uses. 

(1)  Research  Tools.  The  Web  offers  many  kinds  of  research  tools. 
Search  engines,  government  data  sites,  sites  by  subject-matter  experts,  and  professional 
resources  are  just  as  few.  Examples  are  sites  by  National  Aeronautics  Administration, 
United  States  Geological  Survey,  and  other  agencies. 

(2)  Syllabus  Assignment  Listings.  This  is  the  simplest  and  least 
interactive  use  of  the  course  Web  page.  Provided  are  a  copy  of  the  course  syllabus, 
assignments,  contact  information,  perhaps  a  photograph  of  the  professor  and  a  short 
biography,  if  desired.  The  syllabus  can  be  updated  as  often  as  necessary  to  keep  it  current. 
A  list  of  discussion  questions  could  be  added  so  those  students  could  keep  them  in  mind 
while  reviewing  the  last  class  or  preparing  for  the  next.  This  kind  of  page  is  fairly  easy  to 
create  and  maintain  with  minimum  effort. 

b.  Moderately  Interactive  Use  of  the  Internet 

Moderately  interactive  use  of  Internet  has  a  conservative  degree  of 
interactivity  between  the  student,  instructor,  or  expert  at  the  distant  locations.  Instructors 
and  experts  may  act  as  facilitators;  and  they  can  provide  support,  feedback,  and  guidance 
mostly  via  asynchronous  communication,  like  e-mail,  which  allows  for  time-independent 
interaction. 

(1)  Newsgroup-like  Discussion  Groups.  Discussion  groups  can  be 
set  up  so  that  the  professor  can  post  questions  for  the  students  to  discuss  via  e-mail.  Each 
response  is  sent  to  a  server  which  publishes  the  message  in  a  list  for  other  students  to  view. 
These  groups  can  be  made  private  so  that  only  students  enrolled  in  the  course  has  access  to 
the  lists.  If  large  classes  make  in-class  discussions  impractical  or  if  anonymous  discussion 
is  desired,  then  a  Web-based  group  discussion  may  be  a  useful  addition  to  a  course  Web 
site. 
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(2)  Online  Chat  Sessions.  These  sessions  occur  in  real  time,  which 
means  that  as  long  as  at  least  two  people  are  logged  in,  a  live  discussion  can  take  place. 
When  one  participant  types  a  message,  all  other  participants  see  the  message  at  once.  These 
sessions  can  be  embedded  directly  in  a  Web  page  or  accessed  with  special  software 
designed  for  the  purpose. 

(3)  Quizzes,  Surveys  and  Appointment  Books.  Forms  can  be  added 
to  a  Web  site  with  a  little  effort.  An  example  of  a  form  is  a  quiz  that  each  student  fills  out 
and  submits  electronically.  It  can  be  e-mailed  to  the  professor,  automatically  graded,  and 
added  to  the  database  of  that  student's  work. 

c.  Highly  Interactive  Uses  of  Internet 

Highly  interactivity  use  of  the  Internet  occurs  when  NBI  students  can 
interact  with  each  other,  with  instructors,  and  on-line  resources  in  real  time.  Instructors  and 
experts  may  act  as  facilitators.  They  can  provide  support,  feedback,  and  guidance  via 
synchronous  communication,  like  conferencing  tools,  which  allows  for  live  interaction. 

(1)  Real  Time  Interaction  With  Others.  Real  time  interaction  can  be 
done  on  the  Web  in  several  ways.  One  is  to  have  a  link  that  opens  another  application, 
which  makes  a  connection  with  a  server.  Two  types  of  real  time  interaction,  which  are 
accessed  this  way,  are  Internet  Relay  Chat  (IRC)  and  Multi  User  Object  Oriented  (MOO). 
IRC  and  MOO  allow  virtual  gathering  of  students  and  teachers  from  around  the  globe  at  a 
set  time  to  discuss  a  set  of  topics. 

As  technical  advances  make  resources  more  accessible,  the  Web  will 
be  a  viable  medium  to  facilitate  learning.     NBI  has  the  ability  to  provide  a  learning 
environment  that  is  global  and  interactive.   This  technology  can  make  it  easier  and  quicker 
for  organizations  to  train  and  educate  their  personnel  anytime  and  anywhere.  [Ref  6] 
4.  Learning  Styles 

Distance  learning  may  extend  access  to  learning  opportunities  to  students  who 
otherwise  might  not  be  served.    However,  options  that  merely  replicate  the  problems  and 
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failures  of  the  conventional  classroom  will  not  benefit  students.  To  create  options  that 
enhance  learning,  we  must  consider  different  learning  styles  in  the  context  of  distance 
learning. 

A  Web-based  environment  can  accommodate  flexibility  in  learning  styles.  The  way 
an  individual  learner  reacts  to  the  overall  learning  environment  makes  up  his  or  her  learning 
style.  No  universally  accepted  terminology  exists  to  describe  learning  style  and  its  various 
components;  however,  that  people  react  to  their  learning  environment  is  a  core  concept. 
James  and  Blank  (1993)  define  learning  style  as  the  complex  manner  in  which  learners  most 
efficiently  and  effectively  perceive,  process,  store,  and  recall  what  they  are  attempting  to 
learn.  [Ref.  14]  Learning  style  has  three  distinct  but  interconnected  dimensions:  the 
perceptual  (physiological  or  sensory)  mode,  the  cognitive  (mental  or  information 
processing)  mode,  and  the  affective  (emotional  or  personality  characteristic)  mode. [Ref.  5] 

a.  Perceptual  dimension 

Perceptual  dimension  identifies  the  way  individuals  assimilate  information. 
It  involves  the  body's  biological  response  to  external  stimuli,  including  input  through 
physiological  factors  such  as  speech,  movement  and  any  of  the  five  senses.  This  dimension 
depends  on  the  physical  ability  of  an  individual's  body  to  integrate  information  into  the 
brain.  It  is  the  means  through  which  the  information  is  extracted  from  the  environment 

b.  Cognitive  dimension 

The  cognitive  dimension  includes  the  storage  and  retrieval  of  information  in 
the  brain.  Information-processing  habits  represent  the  learner's  typical  way  of  perceiving, 
thinking,  problem  solving,  and  remembering.  Each  learner  has  preferred  avenues  of 
perception,  organization,  and  retention  that  are  distinctive  and  consistent. 

c.  Affective  dimension 

Affective  dimension  encompasses  aspects  of  personality  that  relate  to 
attention,  emotion,  and  valuing.  Affective  learning  styles  are  the  learner's  typical  mode  of 
arousing,  directing,  and  sustaining  behavior.    Although  the  components  of  the  affective 
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learning  style  cannot  be  observed  directly,  they  can  be  inferred  from  the  learner's  behavior 
and  interaction  with  the  environment. 

For  the  instructor  developing  a  Web-based  course,  the  challenge  is  to 
analyze  the  learning  style  characteristics  of  the  group  and  develop  modes  of  instruction  that 
fit  those  characteristics.  Unfortunately,  instructors  are  only  able  to  identify  a  few  elements 
of  their  students'  learning  styles  through  observation.  Consequently,  instructors  should 
adapt  instruction  to  the  variety  of  ways  individuals  learn  by  developing  and  planning 
various  tasks  that  cater  to  various  learning  styles. 

Learning  grows  out  of  what  interests  the  learner.     The  Web  offers  an 
excellent  environment  to  support  a  number  of  learning  style  differences.  Faculty  can  create 
a  learning  environment  that  challenges  each  style  of  student  to  pursue  a  topic  further  and 
gather  different  perspectives  on  issues.  [Ref.  5] 
5.  Cost  Effectiveness 

The  conversion  of  EME  modules  from  traditional  classrooms  to  NBI  requires  a 
large  investment.  Conversion  requires  an  investment  up  front  while  resident  training 
programs  continue;  payback,  in  the  form  of  travel  and  per  diem  cost  savings,  begins  as  NBI 
comes  online  and  replaces  resident  courses.  Such  an  investment  is  difficult  to  defend 
without  considering  the  costs  and  benefits  of  converting  EME  modules  to  NBI. 

Two  of  the  greatest  potential  benefits  are  increased  training  effectiveness  and  the 
saving  of  travel  and  per  diem  costs  for  resident  training.  Training  effectiveness  is  a 
complex  outcome  measure.  It  includes  the  ability  of  NBI  to  provide  at  least  the  same 
amount  of  knowledge  and  as  well-educated  of  a  learner  compared  to  the  traditional 
classroom,  quicker  ability  to  update  course  materials,  and  increase  throughput  of  the 
number  of  students.  [Ref.  1 2] 

A  Naval  officer  thesis  study  on  Cost  Effectiveness  Analysis  of  Converting  a 
Classroom  Course  to  a  Network  Based  Instruction  determined  that  converting  only  two 
EME  modules  to  NBI  was  not  cost-effective.  [Ref.  1 5]  The  conversion  of  more  modules, 
however,  could  generate  cost  savings. 
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Modules  presented  during  the  first  week  of  EME  conference  were  of  varying  length 
w  ith  some  modules  lasting  for  two,  three  or  four  hours.  On  the  second  and  the  third  weeks 
of  the  EME  conference,  each  day  was  broken  into  two,  4-hour  course  modules  with  one 
module  in  the  morning  and  the  second  in  the  afternoon.  The  study  was  based  on  the  use  of 
two,  4-hour  modules  per  day.  If  two  modules  are  converted,  one  fewer  EME  conference 
day  is  required  and  its  costs  are  eliminated.  The  cost  per  student  to  present  one  EME 
conference  is  $13,  348.37.  The  total  cost  per  student  actually  increases  for  each  module 
converted  to  NBI,  until  four  EME  conferences  have  been  held.  Only  when  20  modules  are 
converted  to  NBI,  and  four  EME  conferences  have  been  held  will  the  cost  per  student  for  an 
EME  conference  with  converted  modules  become  lower  than  the  cost  per  student  for  an 
EME  conference  without  converted  modules.  However,  the  savings  are  minor  at  $142.22 
per  student.  Obviously  as  the  number  of  converted  module  increases,  the  conference  length 
in  days  decreases,  thus  decreasing  the  conference  room  costs,  opportunity  cost  of  student 
salaries,  and  per  diem  costs.  There  is  a  greater  saving  after  the  conversion  of  the  20th 
module,  since  weekend  travel  costs  are  eliminated.  The  savings  continue  to  increase  for 
each  even  number  of  modules  that  are  converted.  If  all  modules  are  converted,  a  savings  of 
$1,234.12  per  student  is  realized.  [Ref.  15] 

Stanford  Technologies  Group,  one  of  the  contractors  currently  transforming  an 
EME  module  to  NBI,  conducted  an  Economic  Analysis  of  Training  Alternatives  consisting 
of  three  cases:  (1)  on-the  job  with  the  instructor  traveling  to  the  student  and  with  no  student 
travel;  (2)  NBI  provided  to  the  student,  no  travel  involved;  and  (3)  centralized  classroom, 
both  students  and  instructors  travel.  Based  on  their  study,  the  total  cost  of  training  strategy 
case  number  two,  based  on  1,000  students,  is  $270,000,  or  $270.00  per  student.  NBI 
instruction  is  $143,600.00  less  expensive  compared  to  case  one  and  $280,000.87  less 
expensive  compared  to  case  three.  Figure  2  shows  the  results  of  the  economic  analysis  of 
training  alternatives. 
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Case  #1 


Case  #2 


Economic  Analysis  of  Training  Alternatives 

Notes  for  using  this  worksheet 

(a)  Update  the  grey  fields  as  needed  for  each  business  case 

(b)  Number  of  tnps  MUST  be  manually  updated 

(c)  Travel  cost  per  instructor  should  consider  the  possibility  of 
multiple  training  sessions,  back-to-back,  at  one  training  site 

(d)  CBT  usually  teaches  the  same  content  in  50-75%  of  the  time 
required  for  classroom,  instructor-led  training 

(e)  Do  not  overlook  packaging  and  shipping  costs  for  student 
training  matenals 


On-the-job 
with  instructor 
travelling  to 
the  student 
No  student 
travel 


IBT  provided 

to  the  student 

No  travel 

involved 


Case  #3 


Centralized 

classroom. 

Both  students 

and  instructor 

may  travel 


Student-Related  Expenses 

Number  of  Students 

1000 

1000 

•  000 

#  Students  per  Class 

4 

1 

12 

U  Training  Sessions  Needed 

250 

1000 

83  33 

Length  of  Training  Session  (in  hours) 

8 

5 

ft 

Travel  Time  to/from  Training  per  Student  (in  hours) 

0 

0 

6 

Student  Salary  per  Hour 

$20  00 

$20  00 

$20  00 

Total  Cost  of  Student  Salanes 

$160,000  00 

$100,000  00 

$280,000  00 

#  of  Tnps  for  Students  to  the  Training  Site  (total) 

0 

0 

1,000 

Travel  Cost  per  Studenl  (including  travel  food  &  lodging) 

$0  00 

$0  00 

S50  00 

Total  Cost  of  Student  Travel 

$0  00 

$0  00 

$50,000  00 

Total  Student-Related  Expenses 

$166,000  00 

$160,600  66 

$330,060  00 

Instructor-Related  Expenses 

#  Training  Sessions 

250 

1000 

83  33 

Length  of  Training  Session  (in  hours) 

8 

5 

8 

#  Instructors  per  Training  Session 

• 

0 

2 

Initial  Instructor  Prep  Time  (per  instructor,  in  hours,  not  mc 
matenal 

40 

0 

40 

Instructor  Preparatory  Time  per  Session  (per  instructor,  in  hours) 

2 

0 

4 

Instructor  Salary  per  Hour 

$40  00 

$40  00 

$40  00 

Travel  Time  to/from  Training  Site  per  Instructor  (in  hours) 

8 

0 

8 

Total  Cost  of  Instructor  Salary 

$181.600  00 

$0  00 

$136,533  33 

#  of  Tnps  per  Instructor  to  the  Training  Sites  (total) 

100 

100 

40 

Travel  Cost  per  Trip  per  Instructor  (including  travel,  food  & 
lodging 

$500  00 

$0  00 

$200  00 

Total  Cost  of  Instructor  Travel 

$50,000  00 

$0  00 

$16,000  00 

Total  Instructor-Related  Expenses 

S231  600  00 

SO  00 

$152,553  33 

Other  Related  Expenses 

U  Training  Sessions  Needed 

250  00 

1000  00 

83  33 

Cost  of  Classroom  per  Session 

$0  00 

$000 

$250  00 

Total  Cost  of  Classrooms 

$0  00 

$0  00 

$20.833  33 

Cost  of  Equipment  per  Student 

SO  00 

$0  00 

$25  00 

a  Students 

1000 

1000 

1000 

Total  Cost  of  Equipment 

0 

0 

25000 

Sunk  Cost  to  Develop  Student  Matenals  (Workbooks,  CBT,  etc  ) 

$10.000  00 

$150,000  00 

$10.000  00 

Cost  per  Student  for  Matenals  (Copies.  Replication,  Shipping, 
etc) 

$1200 

$20  00 

$12  00 

Total  Cost  of  Student  Matenals 

$22.000  00 

$170,000  00 

$22,000  00 

Total  Other  Related  Expenses 

$22.66o  66 

$176,666  00 

$67,833  33 

S"ummary  of  Costs 

Total  Student-Related  Expenses 

S160  000  00 

$100,000  00 

$330  000  00 

Total  Instructor-Related  Expenses 

$231  600  00 

$0  00 

$152.533  33 

Total  Other  Related  Expenses 

$22,000  00 

$170,000  00 

$67,833  33 

Total  Cost  of  Training  Strategy 

$413,600  00 

$270,000  00 

$550,366  67 

#  Students 

1000 

1000 

1000 

Cost  per  Student 

$413  60 

$276  66 

$550  37 

Figure  2.  Economic  Analysis  of  Training  Alternatives 

Although  the  sample  analysis  conducted  by  Stanford  is  completely  constructive,  just 
bear  in  mind  that  the  data  used  were  only  approximate.  One  example  is  the  data  they  used 
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for  instructor's  salary,  which  is  $40.00  per  hour,  while  the  instructor's  at  NPS  may  be  as 
high  as  $300.00  per  hour.  The  $270,000  total  training  cost  for  case  two  is  really  coming 
from  the  sunk  cost  of  $150,000  that  is  used  to  develop  student  course  materials.  The 
question  is.  What  are  they  getting  with  this  $150,000?  If  they  will  get  an  engaging,  robust 
distributive  learning  module,  then  the  answer  may  be  yes.  However,  if  they  will  get  a 
totally  asynchronous  text  page  turning  module,  even  when  the  dollar  value  per  student  is 
down  that  may  not  be  the  correct  pedagogy.  Everything  must  be  taken  into  consideration 
when  evaluating  alternatives.  If  they  are  just  looking  to  reduce  cost  as  their  bottom  line, 
then  they  will  look  for  the  cheapest  way  to  get  the  learning  out  to  the  students.  The 
resulting  product  may  not  be  good,  their  credibility  goes  down,  and  learners  are  not  getting 
the  required  education.  However,  if  they  are  looking  for  the  quality  of  education,  they  may 
be  willing  to  spend  the  same  amount  of  money  per  student  and  be  able  to  provide  to  twice 
as  much  the  number  of  students. 

H.         SUMMARY 

As  the  Information  Age  evolves  and  technical  advances  make  resources  more 
accessible,  NBI  is  becoming  a  more  viable  medium  to  facilitate  learning.  Several 
organizations,  including  the  DoD,  schools,  and  private  companies  have  adapted  to  this 
changing  technology.  NBI  make  it  easier  and  quicker  for  organizations  to  train  and  educate 
their  personnel  anytime  and  anywhere.  However,  conversion  of  EME  modules  to  NBI 
requires  a  large  investment  up-front.  Conversion  of  more  modules  will  reduce  the  number 
of  training  days,  resulting  in  greater  savings.  The  potential  benefits  are  increased  education 
or  training  effectiveness  and  savings  of  travel  and  per  diem  costs  for  resident  training. 
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III.  DECISION  PROCESSES  IN  THE  CONVERSION 

A.         INTRODUCTION 

This  chapter  describes  the  initial  decision  process  used  in  the  conversion  of  an  EME 
module  to  NBI.  The  focus  will  be  on  the  criteria  used  by  IDEA  in  the  selection  of  modules, 
instructors,  and  contractors. 

This  pilot  project,  the  first  time  that  an  EME  module  will  be  converted  to  NBI. 
seeks  to  improve  the  current  process  so  that  future  conversion  of  modules  will  be  more 
effective. 

1.  Background 

The  Naval  School  of  Health  and  Sciences  (NSHS)  is  a  BUMED  command  in  charge 
of  the  management  of  the  EME  Program.  The  Institute  for  Defense  Education  and  Analysis 
(IDEA),  a  department  within  the  office  of  the  Associate  Provost  for  Innovation,  is  the 
designated  coordinator  of  the  EME  Program. 

On  April  30,  1997,  IDEA  submitted  a  proposal  to  study,  implement,  and  evaluate 
distance  learning  applications  for  EME  to  the  Commanding  Officer,  NSHS.  This  proposal 
included  empirical  research,  modular  conversions,  measure  of  effectiveness  and  delivery  of 
the  plan.  [Ref.  16]  When  it  submitted  this  proposal,  IDEA  had  no  established  criteria  for 
the  selection  of  modules,  instructors,  and  vendors  for  the  conversion  to  NBI. 

2.  Institute  for  Defense  Education  and  Analysis  (IDEA) 

IDEA  is  charged  with  developing  strategic  partnerships  with  select  universities, 
government  agencies,  and  private  organizations  to  foster  a  mutually  supportive  environment 
for  innovative  collaboration  in  the  design,  creation,  and  implementation  of  an  effective 
distributed  learning  systems. 
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a.  Mission 

IDEA's  mission  is  to  identify,  develop,  market,  and  manage  innovative 
programs  of  applied  research,  consultation,  and  teaching,  including  non-residential 
executive  and  continuing  programs,  on  a  cost-reimbursable  basis,  in  support  of  the  DoD 
requirements. 

IDEA's  educational  programs  are  designed  to  conform  with  the  NPS 
resident  Graduate  Curricula.  They  contribute  to  NPS  faculty/student  development, 
instruction,  and  research  and  build  upon  the  NPS  core  mission.  The  NPS  mission  is  to 
increase  the  combat  effectiveness  of  United  States  and  Allied  armed  forces  and  enhance  the 
security  of  the  United  States  of  America  through  advanced  education  and  research  programs 
focused  on  the  technical,  analytical,  and  managerial  tools  needed  to  confront  defense-related 
challenges.  IDEA's  programs  contribute  to  educational  and  training  innovation,  including 
distance/distributed  learning  and  non-traditional  educational  methods. 

b.  Organization 

The  Associate  Provost  for  Innovation  serves  as  the  Director  of  IDEA. 
Currently,  three  organizational  subdivisions  and  two  centers  support  IDEA'S  mission: 
Education,  Applied  Research,  and  Consultative  Services;  and  The  Center  for  Acquisition 
Education,  Training,  and  Research  and  the  Naval  Center  for  Acquisition  for  Education. 
Figure  3  shows  IDEA's  organizational  structure.  Its  staff  consists  of  NPS  professors  and 
civilian  professionals  who  are  experts  in  their  fields.  Currently,  IDEA  has  limited 
programming  capability,  and  has  no  immediate  plans  to  establish  an  in-house  development 
capability. 

c.  Core  Competencies 

IDEA's  core  competencies  include:  (1)  development  and  delivery  of 
executive  education  programs;  (2)  distributed  learning  /network-based  instruction  course 
conversion  and  development;  (3)  consulting  and  applied  research  program  development 
and  support;     (4)  program  and  project  management;  (5)  conference  coordination  and 
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logistics  support;  and  (6)  survey  development  and  execution,  both  automated  and  traditional 
solutions. 

Office  of  the  Associate  Provost  for  Innovation 


Superintendent 


Provost 


Associate  Provost 
for  Innovation 
(OI) 

Deputy  (  OI-D) 


Institute  for  Defense 
Education  and  Analysis 
(OI-I) 


Education 
Programs 
Group 
(OI-IE  ) 


Applied 

Research 

Programs 

Group 

(OI-IR ) 
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Programs 
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(OI-IC  ) 


Organizational  Support 
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Center  for  Acquisition 

Education  Training 

&  Research 

(OI-CAETR  ) 


Naval  Center  for 
Acquisition 
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Figure  3.  IDEA'S  Organizational  Structure 
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B.         NBI  MODULE  SELECTION  PROCESS 

The  selection  process  for  the  selection  of  modules,  instructors,  and  vendors  starts 
with  the  Distributive  Learning  Team.  The  team  is  made  up  of  four  to  five  members,  who 
are  faculty  and  staff  members  of  NPS.  Their  responsibility  includes:  (1)  oversee  and 
manage  the  delivery  of  residential  and  distributed  graduate  education,  (2)  create  or  adapt 
content  to  new  distributed  learning  pedagogies,  (3)  explore  new  instructional  system 
technologies  to  foster  collaborative  learning  environments  (synchronous  and 
asynchronous),  (3)  provide  analysis  in  determining  course/class  attributes,  (4)  facilitate 
customer  support/interface  between  facilities,  contractors,  and  subject  matter  experts,  and 
(5)  conduct  cost-benefit  analysis  of  alternatives.  The  team  will  identify  and  evaluate  the 
soundness  of  alternative  choices  for  module,  instructor  and  contractor  selection,  and  submit 
their  recommendation  to  the  Director  and  Executive  Director  of  IDEA  for  the  final  decision. 
[Ref.  17] 

1.  Proposal  Development 

The  initial  decision  process  to  convert  current  EME  modules  to  NBI  started  when 
IDEA  submitted  its  proposal  to  NSHS.  This  proposal  laid  out  the  deliverables,  which 
focused  on  two  distance-learning  alternatives:  video  teleducation  (VTE)  and  NBI 
interactive  remote  learning.  Also  explained  were  the  contents,  methodology,  strategy  and 
budget  requirements  for  these  deliverables.  [Ref.  16] 

The  proposal  to  convert  EME  modules  to  NBI  was  approved  and  funded  by 
BUMED  for  $2.5  million.  This  funding  includes  traditional  classroom  instruction  and 
conversion  of  modules  to  NBI  and  VTE.  [Ref.  18]  The  Distributive  Learning  Team, 
formed  to  jump-start  this  initiative,  was  given  the  task  of  selecting  modules,  instructors,  and 
vendors  to  start  the  conversion  process. 

2.  Module  selection 

Different  EME  modules  have  different  module  content.  The  two  modules  selected 
for  the  pilot  study  were  Module  24  (Budgeting  for  Defense  Health)  and  Module  51 
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(Medical  Readiness).  According  to  the  Executive  Director  of  IDEA,  "Those  two  modules 
were  selected  because  they  are  tightly  developed,  well  written  and  organized.  Also,  the 
modules  lasted  only  for  a  short  period  of  time,  no  more  than  four  hours. "  [Ref.  1 7]  The 
EME  Program  Coordinator  of  IDEA  said  that,  "One  of  the  criteria  that  was  used  was  the 
study  conducted  by  Crawford  and  Suchan,  "Media  Selection  in  Graduate  Education/'  [Ref. 
19]  Although  NBI  was  not  included  in  Crawford  and  Suchan' s  media  selection  study, 
according  to  the  author,  VTE's  use  of  50  percent  or  more  on  Know  and  Supply  Information 
and  Apply  Information  to  Structured  Process  Learning  Outcomes  can  be  apply  to  NBFs. 
Also,  according  to  the  survey  Crawford  and  Suchan  conducted,  those  two  modules  do  not 
require  interpersonally  complex  interaction  to  attain  their  learning  outcome,  which  made 
them  excellent  candidates  for  conversion  to  NBI.  [Ref.  20] 

Many  see  the  WWW  as  the  next  great  alternative  to  traditional  instruction. 
According  to  Romiszowski,  "Given  recent  developments  in  telecommunications 
technology,  NBI  can  replicate  virtually,  all  the  key  learning  activities  that  occur  in 
traditional  classroom-based  courses."  [Ref.  21]  Although  EME  modules  can  be  converted 
to  NBI,  the  question  is  how  well  can  they  be  converted?  Those  that  require  a  significant 
amount  of  personal  live  interaction  may  be  extremely  difficult  to  transform  to  NBI.  While 
there  are  limitations,  as  with  other  distance  education  technologies  in  the  degree  to  which  it 
can  replicate  traditional  instructional  events,  NBI  can  be  an  enhancement  over  traditional 
classroom-based  and  distance  education  environments. 

3.  Instructor  Selection 

Selection  of  the  most  qualified  professor  is  very  important  for  the  success  of  this 
pilot  program  since  he  or  she  will  provide  the  course  content  of  the  module.  Although  there 
are  many  excellent  professors  at  NPS,  Professor  Doyle  was  selected  to  convert  his  modules 
to  NBI  based  on  his  reputation  and  accomplishments.  He  was  the  1996  recipient  of  the 
prestigious  RADM  John  J.  Schieffelin  Award  for  excellence  in  teaching.  Well  respected  by 
students,  staff,  and  faculty,  he  is  also  motivated  and  open  to  new  instructional  design  ideas. 
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In  addition,  he  is  computer  literate,  especially  comfortable  with  the  use  of  Power  Point  for 
presentations.  [Ref.19] 

4.  Contractor  Selection 

To  design  Web-based  courses,  design  teams  are  needed.  The  purpose  of  the  design 
team  is  to  unite  a  strong  variety  of  experiences  and  perspectives,  with  the  educational 
learning  outcomes  as  the  central  impetus  and  innovative  technologies  such  as  NBI  as  the 
medium.  The  objective  of  the  design  team  is  to  create  educational  content  that  is  of  the 
highest  caliber  and  to  deliver  it  using  the  most  relevant  technologies  available.  The  design 
team  is  comprised  of  discipline-specific  experts,  Web-design  professionals,  program 
administrators,  standards-based  education  consultants,  and  educational  technologists. 
Currently,  IDEA  does  not  have  a  Web  design  team  to  transform  modules  to  NBI.  In  order 
to  accomplish  this  conversion,  they  needed  to  contract  for  a  Web  design  team. 

The  Executive  Director  of  IDEA  stated,  "We  do  not  have  an  established  criteria  or 
decision  process  in  the  selection  for  contractor  or  vendor."  This  will  be  the  first  time  that  an 
EME  course  module  will  be  transformed  to  NBI  at  NPS.  The  management  of  IDEA  made 
the  decision  to  convert  two  EME  modules  to  NBI,  with  funding  authorization  from 
BUMED.  One  of  their  purposes  for  conducting  this  pilot  study  is  to  learn  and  establish 
criteria  for  selection  of  contractors,  modules,  instructors,  and  other  requirements.  Their 
goal  is  to  learn  how  to  effectively  coordinate  different  NBI  components  and  processes, 
which  include  the  following:  (1)  content  development,  (2)  multimedia  components,  (3) 
Internet  tools,  (4)  computers  and  storage  devices,  (5)  connections  and  service  providers,  (6) 
authoring  programs,  (7)  servers,  and  (8)  browsers  and  other  applications.  [Ref.  1 7] 

IDEA  started  the  contractor  selection  process  by  contacting  approximately  a  dozen 
contractors  to  make  a  presentation  or  demonstration  of  their  products.  There  were  eight 
contractors  who  made  presentations.  Two  contractors  were  selected,  Stanford  Technologies 
Group  and  Arista  Systems,  Incorporated,  because  they  were  able  to  demonstrate  to  IDEA 
management's  satisfaction  that  they  were  capable  of  converting  EME  modules  to  NBI. 
[Ref.  17] 
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Stanford  Technologies  Group  (STG),  based  in  Washington  D.C.,  was  selected 
because  of  their  prior  military  training  development  experience.  However,  because  they  do 
not  have  experience  in  developing  courseware  products,  they  established  a  partnership  with 
Electronic  Learning  Facilitators  (ELF),  Inc.  to  provide  computer  services  including 
analysis,  design,  and  courseware  for  NBI  modules  to  win  the  contract.  The  company 
demonstrated  a  NBI  training  module  of  how  to  play  a  woodwind  instrument  and  conducted 
an  Economic  Analysis  of  Training  Alternatives.  Based  on  this  work,  they  were  assigned  to 
do  the  conversion  of  Module-24,  Budgeting  for  Defense  Health.  There  was  no  specific 
reason  why  IDEA  selected  STG  to  transform  Module-24;  according  to  the  Executive 
Director,  IDEA,  "We  just  want  to  assign  a  module  to  each  vendor."  [Ref.  1 7] 

Arista  Systems,  Inc.,  the  other  contractor  selected,  is  based  in  Monterey  County.  It 
has  partnerships  and  alliances  with  California  State  University  Monterey  Bay,  The 
Laurasian  Institution,  and  Nousoft,  Inc.  Arista  has  experience  with  distance  technology 
through  its  partnership  with  Lauratian  Institution.  To  win  the  contract,  they  were  able  to 
show  a  course  module  via  the  Internet,  titled  "Doing  Business  in  Japan."  That  was  one  of 
several  modules  that  they  had  developed  for  the  Asian  Pacific  Economic  Countries 
sponsored  by  the  Virtual  University  of  Asia.  Arista  is  also  putting  together  an  Accredix 
Platform  Neutral  Course  Management  System,  which  supports  the  delivery  of  instructional 
material  and  includes  the  management  of  network-based  environment.  Modules  that  are 
converted  by  other  contractors  can  be  used  within  this  system.  Another  reason  for  Arista's 
selection  was  its  Monterey  location,  making  the  company  more  accessible  during  the 
conversion.  Arista  was  assigned  to  do  Module  5 1 ,  Medical  Readiness. 

C.         SUMMARY 

IDEA  did  not  use  a  structured  decision  process  or  specific  criteria  to  select  modules, 
instructors,  and  contractors  in  the  conversion  of  an  EME  module  to  NBI.  Their  goal  was  to 
learn  from  this  pilot  study  and  to  establish  criteria  and  a  process  for  future  conversion 
projects. 
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The  next  chapter  will  offer  recommended  criteria  on  the  selection  process  and  assess 
the  decision  process  used  by  IDEA  compared  to  other  organizations. 
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IV.      ASSESSMENT  OF  THE  DECISION  PROCESSES 

A.  INTRODUCTION 

This  chapter  assesses  the  decision  process  used  by  IDEA  in  transforming  an  EME 
module  to  NBI  and  provides  recommended  criteria  for  the  selection  of  module,  instructor 
and  vendor.  It  will  also  discuss  processes  used  by  other  organizations  in  their  conversion 
projects  to  serve  as  a  model  for  the  conversion  process  and  provide  insight  on  how  to  plan 
the  conversion  process. 

B.  THE  DECISION  PROCESS 

The  decision  to  develop  and  maintain  a  new  instructional  system,  such  as  the 
conversion  of  EME  modules  to  NBI,  is  a  large  undertaking.  This  undertaking  should  be 
based  on  a  reasonable  set  of  probable  outcomes  and/or  benefits.  IDEA  management  and  the 
Distributive  Learning  Team,  as  the  managers  of  this  project,  are  entrusted  with  the 
challenge  of  planning  for  this  kind  of  change.  The  new  instructional  system  they 
implement  must  ensure  that  learners  develop  the  new  knowledge,  skills,  and  attitudes 
required  to  meet  program  objectives.  To  meet  program  objectives,  IDEA  management  and 
team  members  should  use  models  such  as  the  Instructional  Systems  Development  Model  to 
plan,  develop,  and  deliver  new  systems  to  deliver  educational  and  training  programs. 

1.  Instructional  Systems  Development 

Instructional  Systems  Development  (ISD)  procedures  have  had  an  impact  on  the 
way  education  and  training  programs  are  planned,  developed  and  delivered.  ISD  has 
military  beginnings.  The  landmark  event  in  ISD  was  the  Inter-service  Procedures  for 
Instructional  Systems  Development  Project  in  1975,  which  offered  a  model  for  using  ISD  in 
the  military.  The  ISD  model  also  influenced  the  development  of  educational  and  training 
programs  in  business  and  industrial  settings.    [Ref.  22]    Although  the  ISD  model  is  used 
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generally  to  systematically  develop  training  programs,  this  model  can  be  used  profitably  to 
develop  new  instructional  technologies. 

Principles  of  ISD  are  more  applicable  when  the  expected  outcomes  of  the  education 
or  training  can  be  clearly  established,  and  the  organization  providing  the  education  is  held 
accountable  for  producing  results.  ISD  projects  flourish  in  organizations  that  place  an 
economic  value  on  the  time  of  its  learners. 

ISD  is  not  a  remedy  for  all  education  and  training  problems;  nor  is  it  guaranteed  to 
be  equally  effective  in  all  situations.  The  purpose  of  ISD  is  to  assist  in  fulfilling  the 
requirements  for  the  successful  implementation  of  educational  and  training  programs. 

All  ISD  models  have  several  things  in  common.  First,  they  are  process-oriented  and 
not  product-oriented.  All  too  often,  programs  are  implemented  without  a  thorough  analysis. 
This  model  will  help  the  decision-makers  to  make  the  right  decision  in  developing  and 
managing  educational  programs.  Second,  the  ISD  approach  is  systematic  and  deliberately 
planned.  Third,  all  ISD  models  have  five  stages,  each  with  a  similar  name  to  describe  a 
similar  purpose.  Fourth,  output  from  one  phase  acts  as  input  for  the  next.  Finally,  this 
process  has  built-in  controls  for  review  and  required  revision. 

The  intent  of  this  ISD  model  is  to  present  information  about  the  process  one  goes 
through  in  developing  instructional  systems,  rather  than  a  fixed,  step-by-step  sequence  of 
specific  actions.  This  model  will  be  used  to  assess  the  decision  process  used  by  IDEA  and 
other  organizations  in  the  conversion  of  their  courses  and  modules  to  NBI.  The  five  stages 
of  the  ISD  model  are: 

a.  Analysis  Phase 

The  process  begins  with  the  Analysis  Phase,  in  which  the  educational  or 
training  needs  of  the  different  groups  of  students  within  the  organization  are  analyzed.  This 
phase,  one  of  the  most  complex  and  time-consuming,  begins  with  few  or  no  assumptions 
about  what  education  or  training  is  required.  This  phase  defines  the  problem,  identifies  the 
source  of  the  problem,  and  determines  possible  solutions.  The  phase  may  include  specific 
research  techniques,  such  as  need  analysis,  job  analysis,  and  task  analysis.   The  outputs  of 
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this  phase  often  include  the  instructional  goals  and  a  list  of  tasks  to  be  taught.    These 
outputs  will  serve  as  the  inputs  for  the  design  phase. 

b.  Design  Phase 

The  Design  phase  involves  using  the  outputs  from  the  analysis  phase  to  plan 
the  strategy  for  developing  the  instruction.  This  phase  develops  an  outline  for  reaching  the 
instructional  goals  determined  during  the  analysis  phase  and  expands  on  the  instructional 
foundation.  Objectives  are  developed  and  used  to  determine  appropriate  content  and 
methods.  Some  of  the  elements  of  this  phase  may  include  writing  a  target  population 
description,  conducting  a  learning  analysis,  writing  objectives  and  test  items,  and 
sequencing  the  instruction.  The  outputs  of  the  Design  phase  will  be  inputs  for  the 
Development  phase. 

c.  Development  Phase 

The  Development  phase  builds  on  both  the  Analysis  and  the  Design  phases. 
The  purpose  of  this  phase  is  to  generate  the  lesson  plans  and  materials.  This  phase  involves 
the  development  of  the  educational  materials,  which  include  the  content  the  instructor  will 
present,  as  well  as  other  supporting  materials.  Designers  often  work  closely  with  functional 
specialists  who  review  and  even  write  the  first  draft  of  the  materials. 

d.  Implementation  Phase 

The  Implementation  phase  refers  to  the  actual  delivery  of  the  instruction, 
whether  it  is  classroom-room  based,  lab-based,  or  computer-based.  The  purpose  of  this 
phase  is  the  effective  and  efficient  delivery  of  instruction.  It  generally  begins  with  a  pilot 
module,  in  which  materials  and  methods  are  formatively  evaluated  before  they  are  made 
ready  for  mass  use.  This  phase  must  promote  the  students'  understanding  of  material, 
support  the  students'  mastery  of  objectives,  and  ensure  the  transfer  of  knowledge  from  the 
instructional  setting  to  the  job. 
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e.  Evaluation  Phase 

The  final  evaluation  phase  assesses  the  training  at  the  end  of  each 
presentation  to  determine  whether  the  students  have  mastered  the  material  and  whether  the 
instruction  itself  was  appropriate.  Evaluation  should  actually  occur  throughout  the  entire 
instructional  design  process,  within  phases,  between  phases,  and  after  implementation.  A 
second  type  of  evaluation,  which  generally  occurs  some  months  after  the  training,  is 
designed  to  follow  up  on  the  students'1  use  of  what  they  learned  in  order  to  determine 
overall  program  success. 

The  purpose  of  this  five-stage  process  is  to  present  a  model  that  should  be 
used  and  adjusted  according  to  the  users'  unique  environment  and  needs.  Not  everybody 
will  use  it  the  same  way.  Some  may  perform  certain  tasks  simultaneously,  or  in  less  detail, 
or  not  at  all. 

This  ISD  model  can  be  applied  in  the  decision  process  to  convert  EME 
modules  to  NBI.  This  model  can  provide  procedural  frameworks  for  systematically  making 
new  instructional  systems  decisions.  Part  of  this  ISD  process  was  used  to  determine  the 
learning  goals  and  content  of  the  EME  to  be  converted. 

C.         CREATING  AN  INITIAL  DECISION  MAKING  PROCESS  FOR  NBI 

Although  there  are  five  different  phases  in  the  ISD  model,  the  scope  of  this  study  is 
the  initial  decision  process,  which  occurs  mainly  in  the  Analysis  phase.  Figure  4  is  a 
proposed  initial  decision  process  using  the  Analysis  Phase  of  the  ISD  Model  for  conversion 
of  EME  modules  to  NBI.  The  five  step  process  include:  (1)  identify  the  instructional  goals; 
(2)  conduct  needs  assessments;  (3)  establish  criteria  for  module,  instructor,  and  vendor 
selection;  (4)  select  delivery  systems,  module,  instructor,  and  vendor;  and  (5)  plan  the 
project.  [Ref.  22] 

The  first  step  is  the  identification  of  goals.  Usually,  this  is  in  response  to 
educational  and  training  needs.  However,  in  this  case,  the  goal  is  to  convert  traditional 
classroom  EME  modules  to  NBI. 
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Figure  4.  Proposed  Initial  Decision  Process 

The  next  step,  conducting  needs  assessments,  involves  the  identification,  evaluation, 
and  prioritization  of  educational  and  training  needs  to  allow  the  development  of  the  course. 
Part  of  this  approach,  the  ability  of  the  individuals  or  the  team  to  meet  performance 
expectations  for  a  given  job  or  task,  is  analyzed.  It  is  important  to  emphasize  that  no  two 
needs  assessment  are  alike.  Decisions  and  plans  will  vary  with  the  nature  of  the  goals  and 
objectives  of  the  organization. 
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The  third  step  is  establishing  criteria  for  module,  instructor,  and  vendor.  One 
challenge  for  instructional  designers  is  the  task  of  suggesting  the  best  criteria  for  designing 
NBI  course  modules.  Presently,  no  "best  criteria"  for  the  design  of  NBI  exist.  The 
selection  criteria  for  module,  instructor,  and  vendor  suggested  here  are  based  on  the 
literature  describing  other  organizations'  processes. 

In  module  selection,  various  factors  must  be  considered  in  relation  to  the  medium  of 
instruction  to  be  used.  These  include:  type  of  content,  number  and  location  of  students, 
time  available,  instructional  budget,  and  facilities  available.  Matching  module  requirements 
to  technology  is  a  challenging  task  because  of  the  many  variables  involved. 

There  are  several  commercial  tools  available  to  aid  in  module  selection.  One  of 
these,  the  Advisor  2.0,  is  widely  used  and  is  a  candidate  for  standardizing  the  process  of 
media  selection  by  the  U.S.  Air  Force  Education  and  Training  Command.  [Ref.  23]  Also, 
this  is  one  of  the  two  media  selection  tools  being  considered  by  Chief  of  Naval  Education 
and  Training  Command  and  is  now  being  used  by  some  Navy  Training  Commands,  such  as 
Fleet  Training  Center,  San  Diego.  [Ref.  24]  Dr.  J.  Bahlis,  the  author  of  this  program, 
worked  with  the  Air  Force  to  produce  a  program  that  meets  or  exceeds  Air  Force 
requirements.  [Ref.  25]  Used  in  the  support  of  the  recommended  process,  Advisor  is  not 
only  useful  in  media  selection,  but  also  helps  organizations  analyze  costs,  impact,  and  other 
issues  associated  with  evaluating  and  choosing  training.  The  Appendix  is  a  summary  of 
inputs  used  to  generate  a  sample  session  created  by  the  author.  Though  the  sample  course 
"Module  24"  (Budgeting  for  Defense  Health)  is  completely  constructive,  the  corresponding 
EME  course  was  kept  in  mind  as  the  author  followed  Advisor  through  the  step-by-step 
assessment  of  educational  needs.  What  is  immediately  obvious  is  the  programs  format  is 
conducive  to  using  it  as  a  structured  interview  tool.  Bahlis  groups  the  "interview"  questions 
into  four  general  areas,  each  covering  a  different  aspect  of  what  he  calls  "Factors  that 
Impact  Media  Selection." 

The  "Regulations"  section  covers  the  policies  and  procedures  that  are  fundamental 
to  the  course  and  course  environment:  Why  does  the  course  exist?    Is  the  course  subject 
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matter  or  material  classified?  What  type  of  testing  is  required?  The  sample  course  was 
designed  as  a  public  domain  course  which  would  require  the  student  to  demonstrate  skills 
and  knowledge  during  testing. 

The  "Organizational"  section  helps  describe  the  administrative  environment  of  the 
course:  How  much  time  is  available  to  develop  the  course?  What  assets  are  available  to  do 
so?  It  also  extracts  information  about  the  course  objectives,  contents,  and  status. 
Consistency  in  delivery  was  important  and  a  flexible  delivery  schedule  was  required. 

The  *'Training,,  section  of  the  interview  investigates  the  students  and  the  instructors. 
What  are  their  backgrounds?  Where  are  they  located?  Is  it  essential  or  just  desirable  for 
them  to  interact?  Is  the  success  of  the  course  dependent  on  their  ability  to  share  experiences 
with  one  another. 

The  final  part  of  the  interview,  "Instructional  Needs,"  tries  to  capture  what  type  of 
training  environment  can  be  expected:  Are  the  students  computer  literate?  Do  they  have 
access  to  PC's?  The  Internet?  Can  they  travel  to  a  VTT  classroom  or  CBT  center? 

When  the  interview  is  complete,  Advisor  compares  the  requirements  and  matches 
them  to  currently  available  educational  techniques.  Advisor  produces  an  "effectiveness" 
score,  with  100  percent  being  an  ideal  choice.  On  demand,  Advisor  produces  a  list  of 
reasons  why  each  candidate  technology  is  less  than  ideally  suited  for  a  particular  course.  In 
the  sample  case  for  example,  Internet/Intranet  scores  83  percent  because  of  some  deleterious 
environmental  factors  like  student  resistance  to  change.  Understanding  the  logic  underlying 
Advisor  enables  the  user  to  manipulate  the  variables  to  generate  alternative  paths.  In  the 
sample,  a  print-based  was  rejected  outright  because  one  of  the  parameters  selected  was  a 
requirement  for  audio.  If  the  requirement  for  audio  is  negotiable,  a  correspondence  solution 
path  can  be  made  feasible,  with  an  effectiveness  of  60  percent,  by  changing  the  value  of  the 
variable  audio  from  "required"  to  "desired." 

As  discussed  in  Chapter  III,  the  study  conducted  by  Crawford  and  Suchan  on 
"Media  Selection  in  Graduate  Education  for  Navy  Medical  Officers"  demonstrates  an 
approach  for  systematically  selecting  instructional  media  for  specific  learning  applications 
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that  places  priority  on  the  desired  learning  outcome  and  the  media  required  to  support  the 
instructional  techniques  to  attain  that  outcome.  On  this  study,  the  media  selection 
guidelines  were  used  to  match  each  module  with  VTE,  digitized  video  disc  (DVD),  or 
traditional  classroom  instruction.  [Ref.  20]  This  study  can  also  be  useful  in  conjunction 
with  Advisor  2.0. 

The  fourth  step  is  the  establishment  of  selection  criteria  for  the  instructor.  A  critical 
part  of  the  EME  module  conversion  to  NBI  is  the  instructor.  With  the  current  technology, 
NBI  can  be  designed  so  that  the  instructor  plays  the  role  of  facilitator,  coach,  mentor,  and 
guide.  Instructors  are  guided  by  Web  Design  Team  in  specific  methods  and  strategies  for 
facilitating  courses  online. 

To  successfully  facilitate  an  online  course  module,  instructors  must  meet  the 
following  professional  criteria:  (1)  appropriate  content  knowledge  and  experience;  (2) 
competence  with  technologies  required  to  deliver  the  course  successfully —i.e.,  e-mail,  real- 
time communications,  group  mailing  lists;  (3)  ability  to  introduce  and  manage  both 
synchronous  and  asynchronous  communications  and  interactions;  and  (4)  openness  to 
emerging  technologies  and  ways  to  apply  them  to  online  educational  environments.  [Ref. 
26] 

Vendor  selection  criteria  are  especially  important  for  organizations  that  outsource 
their  Web-Design  Team.  Most  distance  education  programs  in  higher  learning  centers,  such 
as  NPS,  do  not  necessarily  have  all  the  expertise  to  perform  design  and  production  tasks. 
The  expertise  lacking  is  most  often  in  Web  programming,  multimedia  authoring,  and  media 
conversion  skills.  A  design  team  is  needed  to  fulfill  those  requirements.  Such  a  team  is 
involved  in  the  design  and  development  of  a  Web-based  module  with  the  objectives  of 
creating  the  highest-caliber  educational  content  and  delivering  it  with  the  most  relevant 
technologies  available.  For  some  organizations,  it  is  financially  prudent  to  contract  their 
conversion  project  out  to  qualified  vendors. 

To  achieve  the  milestones  of  a  Web-based  initiative,  the  following  requirements  and 
criteria  for  vendor  selection  are  suggested:  (1)  vendor  quality  as  evidenced  by  work 
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samples;  (2)  vendor's  price;  (3)  vendor's  understanding  of  the  subject  matter;  (4)  vendor's  ' 
experience  in  the  actual  conversion  of  course  modules  to  NBI;  (5)  quality  of  the  proposed 
solution;  and  (6)  vendor's  '  reliability  and  track  record.  [Ref.  27] 

This  fourth  step,  the  selection  of  module,  instructor,  and  vendors,  must  be  based  on 
the  management  and  Distributive  Learning  Team's  approved,  established  criteria. 

Planning  the  start  of  the  project  is  the  fifth  step,  including  establishing  the  timelines 
and  resolving  other  issues  based  on  the  information  generated  from  the  above  steps. 

D.         ASSESSMENT  OF  OTHER  ORGANIZATIONS'  PROCESSES 

An  assessment  of  the  initial  decision  process  used  by  two  organizations,  the 
University  of  Minnesota  and  Jefferson  County  Public  School,  will  be  discussed  using  the 
Initial  Phase  of  the  ISD  Model.  Its  purpose  is  to  point  out  the  similarities  and  differences  in 
the  steps,  issues,  and  constraints  involved  in  their  decision  process  as  compared  to  IDEA's 
decision  process. 

1.  University  of  Minnesota 

The  following  is  the  assessment  of  the  initial  decision  process  used  by  the 
University  of  Minnesota  in  designing  and  implementing  college-wide  Web-Based  courses: 

a.  Identify  Goals 

In  fall  1996,  the  University  of  Minnesota's  College  of  Education  and  Human 
Development  (CEHD)  administration  asked  the  Web  Development  Team  to  design, 
develop,  and  implement  a  traditional  college  course  for  Web-Based  Instruction.  Their 
purpose  is  to  design  and  maintain  a  standardized  WWW  presence  for  the  college 
representing  communications,  student  services,  and  instructional  uses  of  the  Web.  Also,  to 
create  interest  among  instructors  to  develop  and  manage  their  own  Web-Based  course 
materials.  [Ref.  28] 
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b.  Needs  assessment 

The  management  and  the  present  Web-Based  Design  Team  started  by 
assessing  whether  they  had  the  required  expertise  and  knowledge  for  the  conversion 
process.  Based  on  their  assessment,  they  hired  additional  members  for  the  Web 
Development  and  Design  team  to  jump-start  the  initiative.  Also  included  in  the  team  were 
several  members  of  the  communication  staff,  a  representative  from  the  Technical  and 
Information  Services  unit  of  the  college,  representatives  from  student  support  services,  and 
the  Assistant  Dean  of  the  college. 

c.  Establish  Criteria 

The  University  of  Minnesota  does  not  have  an  establish  criteria  in  the 
courses  and  instructor  selection,  and  vendor  selection  criteria  were  not  applicable 

d.  Selecting  module,  instructor,  and  vendor 

The  team  selected  Course  number  5145  (Social  Studies  and  the  Internet), 
which  is  mainly  a  lecture  course.  Also,  the  instructor  selected  was  knowledgeable  with  the 
use  of  computers  and  motivated  to  work  with  the  team  in  the  conversion  process.  Selecting 
vendor  is  not  applicable  to  this  example  since  they  have  their  own  Web  Development  and 
Design  Team. 

e.  Planning  the  Project 

In  planning  their  project,  the  Design  Team's  tasks  were  broken  into  three 
main  areas:  the  Internet,  the  Intranet,  and  instructional  uses  of  technology.  The  Internet  was 
their  first  priority,  as  the  existing  site  had  been  designed  and  implemented  four  years  before. 
The  initial  redesign  of  the  Intranet  followed,  with  page  graphics  and  templates  designed  to 
complement  the  college  Internet  site.  [Ref.  28] 

/  Other  Issues  and  Constraints 

The  following  constraints  and  issues  were  also  identified  in  addition  to  the 
above  assessment: 
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1.  Lack  of  Funds  and  Bureaucracy:  The  management  rejected  some  of  the  early 
ideas  of  technical  experts.  While  the  design  team  was  busy  resolving  some  of 
the  hardware  problems,  the  conversion  process  was  hampered  by  the  lack  of 
funds,  end-user  hesitation  to  switch  platforms,  and  the  bureaucracy  inherent  in 
any  large  organization. 

2.  Instructors'  Opposition:  Some  instructors  voiced  opposition  to  the  plan  of 
individual  professors  converting  their  own  courses  to  Web-Based  Instruction. 
In  any  group  of  users,  knowledge  on  a  particular  subject  will  vary.  While  some 
of  the  instructors  are  well  versed  in  effectively  using  the  current  technologies, 
while  many  are  poorly  prepared  or  have  no  knowledge  of  the  latest  advances. 
The  Web-Based  Design  Team  started  the  process  of  educating  the  faculty  in  the 
area  of  Internet  development.  They  created  the  World  Wide  Web  toolbox  with 
the  intention  of  providing  a  repository  of  information  relating  to  the 
development  and  use  of  Web  pages  in  the  college.  The  toolbox  was  well 
received,  but  failed  to  generate  significant  interest  from  the  faculty  in  Web- 
based  courseware. 

3.  Incentives  and  Compensation:  Instructors  were  hesitant  to  learn  how  to 
maintain  and  construct  Web  sites  without  some  sort  of  incentive  or 
compensation.  The  management  and  the  team  did  not  take  the  time  to  find  out 
what  incentives  and  compensation  the  instructors  needed. 

4.  Unanticipated  Areas  of  Concern:  The  Web-Design  Team  did  not  anticipate 
several  areas  of  concern,  such  as  Internet  security  and  protecting  copyrights  on 
intellectual  property,  among  others. 

Although  the  University  of  Minnesota  used  an  initial  decision  process,  that 
process  was  inadequate  because  it  failed  to  identify  other  issues,  such  as  funding, 
bureaucracy,  instructors'  opposition  and  areas  of  concern  mentioned  above.  However,  their 
experience  in  converting  their  courses  to  NBI  resulted  in  lessons  learned  that  are  beneficial 
to  their  future  conversion  projects. 

Some  of  the  lessons  learned  are  the  following:  (a)  must  keep  users'  needs  in 
mind  first;  (b)  investigate  what  the  university  is  taking  with  respect  to  Internet  development; 
(c)  consider  the  ability  of  instructors  to  implement  and  support  any  Web  initiative;  and  (d) 
to  start  small  and  work  up  to  a  more  ambitious  as  the  development  initiative  progress. 
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2.  Jefferson  County  Public  Schools 

The  following  is  the  assessment  of  the  decision  process  used  by  Jefferson  County 
Public  School  District: 

a.  Identify  Goals 

The  Jefferson  County  Public  Schools  were  interested  in  introducing  K-12 
on-line  educational  alternatives  into  their  existing  district-wide  curriculum.  Their  intention 
was  .  to  better  serve  students  and  faculty  by  designing  an  alternative  course  that  would 
reach  all  the  populations  served  by  the  district.  [Ref.  26] 

b.  Needs  Assessment. 

Their  first  step  was  to  conduct  a  formal  needs  assessment.  The  assessment 
was  undertaken  with  the  hopes  of  exploring  and  discovering  academic  directions  that  would 
enable  the  district  to  expand  its  educational  vision  and  implementation  and  enable  more 
students  to  successfully  complete  their  high  school  education. 

Based  upon  their  initial  assumptions  and  findings,  they  recommended  the 
introduction  of  NBI  alternatives  as  the  most  appropriate  distance  learning  direction.  They 
also  conducted  a  cost-effectiveness  analysis.  Their  goal  was  to  design  and  deliver  an 
alternative  that  would  benefit  all  the  populations  served  by  the  district. 

As  a  result  of  their  assessment,  they  formed  an  Initial  Design  Team.  Their 
design  team  was  comprised  of  self-recruited  individuals  who  were  able  to  bring  critical 
expertise  to  the  product  development.  These  volunteers  consisted  of  discipline-specific 
experts.  Web-design  professionals,  program  administrators,  standards-based  education 
consultants,  and  educational  technologists.  In  addition  to  the  input  of  people  within  their 
district,  the  school  began  closely  working  with  outside  agencies,  industries,  and 
organizations.  Their  reason  for  initiating  this  working  relationship  was  twofold:  (1)  to 
ensure  that  work-related  realities  were  introduced  into  their  educational  content;  and  (2)  to 
strengthen  their  program  base  for  future  expansion  and  possible  working  partnerships.  This 
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diverse  mixture  of  knowledge  ensured  a  strong  foundation  upon  which  to  build  and  expand 
the  NBI  conversion  initiative. 

c.  Establish  Criteria 

In  this  example,  the  Design  Team  established  and  fulfilled  the  criteria  set 
forth  by  a  number  of  educational  conversion  programs  underway  at  different  levels  in  the 
district.  They  established  the  following  criteria  for  instructor  selection:  (1)  appropriate 
content  knowledge  and  experience;  (2)  competence  with  technologies  required  to  deliver 
the  course  successfully~i.e.,  e-mail,  real-time  communications,  group  mailing  lists;  (3) 
ability  to  introduce  and  manage  both  synchronous  and  asynchronous  communications  and 
interactions;  and  (4)  openness  to  emerging  technologies  and  ways  to  apply  them  to  online 
educational  environments.  [Ref.  26]  Selection  criteria  for  course/module  were  not 
addressed,  and  vendor  selection  criteria  were  not  applicable. 

d.  Selecting  Module,  Instructor  and  Vendor 

The  Administrator  and  the  Web  Design  Team  selected  the  instructor  and 
course  to  convert  according  to  their  established  criteria. 

e.  Planning  the  Project 

During  the  planning  of  their  project,  the  team  identified  the  standards  and 
expectations.  They  were  able  to  outline  the  requirements  for  their  initial  on-line  course 
design  and  development.  It  was  this  step  that  enabled  them  to  know,  from  the  outset,  the 
challenges  ahead  of  them  and  ways  to  go  about  dealing  with  them. 

They  successfully  developed  a  single  Web-based  course.  In  addition,  they 
were  attempting  to  integrate  district  content  standard,  alternative  assessment  formats,  the 
latest  technology,  and  sensitivity  toward  gender  and  culture.  This  is  an  example  of  an  on- 
line program  put  together  thorough  clear  and  cohesive  vision,  careful  planning,  and 
teamwork. 
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E.         ASSESSMENT  OF  THE  IDEA  (NPS)  DECISION  PROCESS 

The  following  are  assessments  of  the  decision  process  used  by  IDEA  in  the 
conversion  of  these  two  EME  modules  to  NBI  using  the  Initial  Phase  of  the  ISD  Model: 

1.  Identify  Goals 

The  NBI  conversion  decision  process  began  when  the  proposal  IDEA  submitted  to 
NSHS  was  approved  and  funded  by  BUMED.  Their  goal  is  to  convert  traditional 
classroom  instruction  to  EME  modules  to  NBI.  This  initiative  will  provide  senior  DoD 
medical  executives  an  opportunity  for  anytime,  anywhere  EME  education  on  demand  on 
both  synchronous  and  asynchronous  environments. 

2.  Needs  Assessment 

The  IDEA  management  and  the  Distributive  Learning  Team  discussed  the  project  in 
detail  and  arrived  at  a  consensus  on  how  the  development  process  would  proceed.  Based  on 
this  assessment,  they  agreed  that  they  did  not  have  the  required  knowledge  and  experience 
to  do  the  conversion  project  in-house  and  recommended  outsourcing  the  Web  Design 
Teams. 

3.  Criteria  for  Selecting  Modules,  Instructors,  and  Vendors 

The  Distributive  Learning  Team  did  not  establish  the  criteria  for  the  selection  of 
modules,  instructors,  and  vendors  for  the  conversion  project.  Without  established  selection 
criteria,  it  was  difficult  for  the  management  and  the  team  to  arrive  at  solid  decisions  in  these 
areas. 

The  management  and  the  Distributive  Learning  Team,  without  previous  experience 
in  this  work,  should  have  conducted  a  more  thorough  assessment  during  this  phase  of  the 
decision  making  process  in  order  to  define  the  problem  and  determine  possible  solutions. 
Part  of  this  phase  is  the  establishment  of  criteria  or  standards  for  selection  of  modules, 
instructors,  and  vendors  prior  to  interviewing  and  selecting  contractors.  Careful  analysis  is 
more  important  when  the  project  is  a  pilot  study,  where  most  of  the  decision-makers  have 
little  familiarity  with  NBI  conversion.    They  should  be  careful  to  select  only  contractors 
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with  proven  track  records  and  capabilities,  who  can  convert  and  produce  an  excellent  EME 
module  to  NBI. 

In  addition,  there  are  no  established  criteria  for  module  selection.  The  criteria  used 
were  from  the  study  by  Crawford  and  Suchan,  "Media  Selection  in  Graduate  Education." 
This  study  was  based  on  VTE  media  selection,  not  NBI.  Many  additional  factors  must  be 
taken  into  account  when  selecting  a  medium  of  instruction:  type  of  content,  number  and 
location  of  students,  time  available,  instructional  budget,  and  facilities  available.  Although 
the  management  and  the  distance  learning  team  have  knowledge  of  these  factors,  and  made 
good  module  selections  using  Crawford  and  Suchan  study;  they  do  not  have  the  educational 
support  tools,  such  as  Advisor  2.0,  to  help  determine  the  feasibility  and  effectiveness  of 
using  several  alternate  media,  including  NBI. 

IDEA  does  not  have  established  criteria  or  standards  for  instructor  selection.  During 
the  selection  process,  the  management  and  the  Distributive  Learning  Team  selected 
Professor  Doyle,  based  mainly  on  his  success  in  teaching,  as  described  in  Chapter  III.  Since 
they  had  chosen  Professor  Doyle,  IDEA  must  also  select  his  modules  for  conversion.  Thus, 
according  to  IDEA,  it  was  a  good  match  of  professor  and  teaching  modules.  This  appeared 
to  be  a  favorable  coincidence.  Presently,  the  process  used  by  IDEA  is  to  select  an  instructor 
first  and  then  select  his  or  her  respective  module.  One  recommendation  is  to  select  the 
module  that  meets  the  criteria  for  conversion  and  is  most  beneficial  for  the  senior  medical 
executives;  and  then  select  the  instructor  assigned  to  the  module  who  meets  the  established 
criteria  for  instructor. 

As  discussed  in  Chapter  III,  IDEA  does  not  have  a  design  team  and  has  limited 
programming  capability  to  convert  EME  modules  to  NBI.  To  achieve  the  conversion  of 
EME  modules,  IDEA  had  no  choice  but  to  outsource  to  a  Web  design  team.  However, 
IDEA  does  not  have  established  criteria  for  selecting  vendors  or  contractors.  They  sub- 
contracted for  their  Web  Design  and  Development  Team  and  selected  the  two  vendors 
discussed  in  Chapter  III.  STG  was  selected  based  on  previous  military  training 
development  experience,  while  Arista  was  selected  based  on  their  actual  NBI  course 
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conversion  experience  and  being  located  in  Monterey  County.  Although  the  two 
contractors  may  be  doing  the  conversion  of  EME  modules  satisfactorily,  a  more  effective 
way  would  have  been  to  establish  criteria  and  standards  first  and  then  select  the  most 
qualified  vendor. 

4.  Planning  the  Project 

In  planning  their  project,  the  IDEA  management  and  Distributive  Learning  Team 
submitted  a  well-written  proposal  to  BUMED.  Their  proposal  outlined  the  phased 
implementation  of  the  distance  learning  applications  within  the  EME  Program.  Also 
included  were  the  plan  of  action,  milestones,  and  deliverables  for  1997-1998.  However, 
some  of  the  issues,  concerns,  and  constraints  on  this  conversion  project  were  not  addressed. 

5.  Other  Issues  and  Constraints 

a.  Distributive  Learning  Team  Performance 

The  process  for  selecting  modules,  instructor,  and  vendors  starts  with  the 
Distributive  Learning  Team.  The  team  is  made  up  of  NPS  faculty  members  and  staff  whose 
job  is  to  support  and  assist  the  management  in  making  a  sound  decision.  One  team  member 
was  not  involved  in  the  decision  process  for  vendor  selection  and  did  not  have  knowledge 
on  why  the  vendors  were  selected.  All  members  must  be  carefully  selected  and  should  be 
included  in  decisions  and  work  as  a  team  in  order  to  be  more  effective  in  the  decision 
making  process.  Their  responsibilities  include:  (1)  overseeing  and  managing  the  delivery  of 
residential  and  distributed  graduate  education;  (2)  creating  or  adapting  content  to  new 
distributed  learning  pedagogies;  (3)  exploring  new  instructional  system  technologies  to 
foster  collaborative  learning  environments  (synchronous  and  asynchronous);  (4)  providing 
analysis  in  determining  course/class  attributes;  (5)  facilitating  customer  support/interface 
between  facilities,  contractors,  and  subject  matter  experts;  and  (6)  conducting  cost-benefit 
analyses  of  alternatives.  The  team  should  have  identified  and  evaluated  the  soundness  of 
alternative  choices  for  modules,  instructors  and  contractors  before  submitting  its 
recommendation  to  the  Director  and  Executive  Director  of  IDEA  for  the  final  decision. 
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b.  Support  for  the  instructor 

The  instructor  doing  the  conversion  experienced  some  problems  in  working 
with  the  contractors,  including  incompatibility  of  the  systems  (Mac  and  IBM),  contractors' 
underestimation  of  the  amount  of  work  to  be  done,  and  the  need  to  deal  with  several  points 
of  contact.  [Ref.  29] 

The  instructor  encountered  these  problems  during  the  early  stages  of 
transformation  process.  Although  these  problems  were  later  resolved,  both  the  professor 
and  the  contractor  lost  valuable  time.  This  could  have  been  avoided  with  careful  advance 
planning,  which  includes  establishing  one  project  manager  for  each  vendor,  checking  on  the 
compatibility  of  NPS's  and  the  vendor's  system,  and  others. 

c.  Funding 

Certainly,  one  constraint  on  this  project  was  funding.  The  decision  to 
implement  a  particular  instructional  program  or  change  an  existing  one  rests  on  identifying 
all  the  costs  of  research  and  development,  personnel,  and  running  the  implementation  for 
the  life  of  the  system,  including  operation  and  maintenance.  Also,  conversion  of  modules 
to  NBI  requires  an  up-front  investment  while  resident  training  continues.  With  the  $2.5 
million  funding  for  the  EME  program,  which  includes  regular  classroom  training  and 
conversion  of  EME  modules  to  NBI  and  VTT,  only  a  small  portion  of  the  funding  is  slated 
for  research  and  development  in  the  conversion  to  NBI.  According  to  IDEA,  BUMED  is 
planning  to  convert  about  28  EME  modules  to  NBI.  On  average,  it  will  cost  $100,000  to 
convert  each  module  to  NBI;  with  28  modules,  this  will  amount  to  $2.8  million.  A  limited 
budget  will  definitely  restrict  further  NBI  conversion  projects. 

d.  Instructors' Opposition 

Another  constraint  is  the  instructor.  Web-based  instruction  may  be  an 
excellent  new  tool  for  learning  and  teaching,  but  faculty  members  face  many  obstacles  to  its 
effective  integration  into  instruction.  Some  faculty  members  are  against  the  conversion  of 
EME  modules  to  NBI  and  the  used  of  other  distance  learning  media.     They  may  be 
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unwilling  to  participate  in  the  conversion  process.  Critical  to  the  future  of  every  tenure- 
tracked  assistant  or  associate  professor  are  promotion  and  tenure  criteria.  Without  a  doubt, 
the  tenure  process  serves  as  a  strong  incentive  for  research  and  an  equal  disincentive  for 
instructional  innovation.  As  a  result,  junior  faculty  might  believe  it  is  not  in  their  best 
interest  to  participate  in  Web-based  instruction  initiatives  unless  promotion  and  tenure 
criteria  reward  instructional  innovation.  [Ref.  30] 

F.         SUMMARY 

The  successful  development  and  maintenance  of  a  new  instructional  system,  such  as 
the  conversion  of  course  modules  to  NBI,  requires  a  systematic  decision  process.  This 
process  should  include  analysis,  design,  development,  implementation,  and  evaluation  of 
the  program.  Criteria  should  also  be  established,  especially  in  the  module,  instructor,  and 
vendor  selection  process.  This  will  eliminate  surprises  and  non-performance. 

Most  first-time  projects  encounter  more  frequent  problems  when  poor  or  inadequate 
decision  processes  are  established  and  no  careful  planning  is  done.  Many  organizations, 
such  as  the  University  of  Minnesota,  Jefferson  County  Public  Schools,  and  NPS,  learned 
lessons  from  their  experience  that  will  be  useful  in  their  future  NBI  conversion  projects. 

IDEA  management  and  the  distributive  team  must  recognize  the  importance  of  a 
comprehensive  decision  process  in  developing  new  instructional  systems.  Although  an 
initial  decision  process  was  used,  it  was  neither  fully  developed  nor  comprehensive.  The 
criteria  for  selection  of  modules,  instructors,  and  vendors  were  not  included  in  their 
assessment.  Additionally,  some  important  issues  such  as  instructor's  compensation  and 
incentives,  intellectual  property  rights  among  others  that  may  affect  the  future  conversion 
process  were  not  addressed. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 

This  research  examined  the  decision  process  behind  the  conversion  of  course 
modules  from  the  Navy  Medical  Department  Executive  Management  Education  curriculum 
to  Network- Based  Instruction.  The  objective  was  to  track  and  model  the  process  of 
integrating  NBI  technology  into  an  existing  traditional  classroom  course  of  instruction.  The 
goal  was  to  capture  the  context  of  the  key  decisions  made  during  the  preliminary  stage  of  an 
actual  conversion  project  and  analyze  the  effectiveness  of  the  approach. 

A.         RESEARCH  QUESTIONS 

1.  What  Process  Was  Used  to  Convert  the  Current  EME  Modules  to  NBI? 

As  discussed  in  Chapter  IV,  IDEA  used  a  process  that  is  not  fully  developed  in  the 
initial  decision  of  converting  current  EME  modules  to  NBI.  This  undertaking  should  be 
based  on  reasonable  expectations  regarding  the  probable  outcomes  or  benefits  from  such  an 
endeavor.  The  IDEA  management  and  Distributive  Learning  Team  should  use  models  such 
as  Instructional  Systems  Development  to  plan,  develop,  and  deliver  education  and  training 
programs. 

2.  Who  Are  the  Stakeholders? 

As  discussed  in  Chapter  II,  a  stakeholder  is  any  group  that  is  affected  by  the 
achievement  of  an  organization's  objectives.  To  be  a  stakeholder  means  to  have  an  interest 
in  the  process  or  outcome  of  the  issue.  Some  stakeholders  may  want  things  to  stay  the 
same;  some  may  favor  small  changes  of  various  sorts;  some  may  want  radical  change. 
There  may  be  stakeholders  who  are  not  involved  in  any  controversy  or  process,  yet  have  an 
interest  in  the  issue.  Active  stakeholders  are  organizations  or  groups  or  occasional 
individuals  who  seek  to  affect  the  current  initiative  because  of  some  interest.  Some  of  them 
may  be  public-spirited  or  acting  on  behalf  of  others. 
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The  EME  initiative  has  six  stakeholders,  not  all  of  whom  are  equally  affected  by  or 
have  a  significant  effect  on  the  initiative: 

1 .  Commanding  Officers  and  senior  military  personnel  are  the  most  significantly 
affected  by  this  initiative.  They  are  required  to  complete  the  EME  curriculum  in 
order  to  acquire  necessary  proficiency  and  demonstrate  professional 
administrative  skills. 

2.  Congress  is  also  an  important  stakeholder  because,  as  the  primary  decision- 
maker, it  issued  a  directive  to  the  DoD  concerning  the  required  skills, 
knowledge,  and  experiences  an  MTF  commander  must  possess. 

3.  Clearly,  DoD,  including  the  individual  military  services,  has  a  major  stake  in 
this  EME  initiative.  The  DoD's  position  is  that  the  training  of  senior  medical 
personnel  is  vital  for  health  care  executives  to  achieve  the  necessary  skills. 

4.  Professors  who  are  experts  in  their  respective  fields  comprise  another  group  that 
can  exert  influence.  The  obvious  impact  of  this  initiative  is  that,  as  more 
medical  officers  go  through  this  program,  more  professors  are  needed  to  teach 
the  required  skills. 

5.  BUMED  has  a  direct  impact  on  the  EME  initiative.  They  are  involved  in  the 
design  and  funding  of  the  EME  program  based  on  the  competencies  identified. 

6.  NPS  Institute  for  Defense  Education  and  Analysis  (IDEA)  is  charged  with  the 
development,  delivery  and  management  of  EME  programs. 

3.  What  Is  the  Perceived  Value  of  the  Conversion  to  NBI? 

The  conversion  of  EME  modules  to  NBI  will  provide  an  opportunity  for  anytime, 
anywhere  EME  education  on  demand  in  both  synchronous  and  asynchronous  environments. 
Also,  successful  conversion  of  EME  modules  to  NBI  may  eventually  become  a  revenue 
generator  for  NPS.  The  potential  benefits  are  increased  training  effectiveness  and  savings 
of  travel  and  per  diem  costs  for  resident  training.  As  discussed  in  Chapter  II,  training 
effectiveness  is  a  complex  outcome  measure.  It  includes  the  ability  of  NBI  to  provide  at 
least  the  same  amount  of  knowledge  and  as  well-educated  of  a  learner  compared  to  the 
traditional  classroom,  quicker  ability  to  update  course  materials,  and  increase  throughput  of 
the  number  of  students. 


46 


4.  What  are  the  Significant  Resources  and  Constraints? 

One  of  the  resources  and  constraints  on  this  project  is  funding.  The  decision  to 
implement  a  particular  instructional  program  or  change  an  existing  one  rests  on  identifying 
all  the  costs  of  research  and  development,  personnel,  and  life  cycle  costs,  including 
operation  and  maintenance.  Also,  conversion  of  modules  to  NBI  requires  an  up-front 
investment  while  resident  training  continues. 

Another  potential  constraints  are  instructors.  According  to  IDEA,  BUMED  plans  to 
convert  28  modules  to  NBI,  finding  all  28  qualified  instructors  could  be  difficult.  Faculty 
members  are  faced  with  many  obstacles  to  its  effective  integration  into  NBI  instruction. 
Some  faculty  members  are  against  the  conversion  of  EME  modules  to  NBI  and  the  use  of 
other  distance  learning  media.  Because  of  the  publishing  demands  of  a  tenure-track 
position  and  other  duties,  it  is  difficult  for  professors  to  spare  the  time  and  help  the  Web 
Design  Team  in  providing  his  or  her  input  to  their  modules'  design,  development,  and 
implementation.  Even  when  the  conversion  of  module  is  being  outsource  to  private 
contractors,  this  still  requires  a  lot  of  instructor's  time  to  work  with  vendor.  Also,  some 
instructors  are  poorly  prepared  and  have  no  knowledge  in  latest  computer  technologies. 
During  the  implementation  stage,  it  would  be  helpful  for  the  instructor  to  possess  a  broad 
range  of  computer  skills  to  deliver  the  course  successfully— i.e.,  e-mail,  real  time 
communication,  and  group  mailing  lists.  If  faculty  members  must  acquire  any  of  these  new 
skills  in  order  to  carry  out  their  vision,  entire  weeks  and  months  can  be  swallowed  without  a 
significant  product  to  demonstrate.  Another  reason  for  instructors  objection  to  EME 
conversion  is  the  lack  of  incentives  and  compensation. 

B.         CONCLUSIONS 

As  the  Information  Age  evolves,  and  technical  advances  make  resources  more 
accessible,  NBI  is  becoming  a  more  viable  medium  to  facilitate  learning.  Several 
organizations,  including  the  DoD,  schools,  and  private  companies,  have  adapted  to  this 
changing  technology.    NBI  makes  it  easier  and  quicker  for  organizations  to  train  and 
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educate  their  personnel  anytime  and  anywhere.  However,  conversion  of  EME  modules  to 
NBI  requires  a  large  investment  up-front.  Transformation  of  more  modules  will  reduce  the 
number  of  training  days,  resulting  in  greater  savings.  The  potential  benefits  are  increased 
training  effectiveness  and  saving  of  travel  and  per  diem  costs  for  resident  training. 

Developing  and  maintaining  a  new  instructional  system,  such  as  the  conversion  of 
course  modules  to  NBI,  requires  a  decision  process.  This  process  should  include  analysis, 
design,  development,  implementation,  and  evaluation  of  the  program.  Criteria  should  also 
be  established,  especially  in  the  selection  process.  This  will  eliminate  surprises  and  non- 
performance. 

C.  RECOMMENDATIONS 

Based  on  this  research,  the  following  recommendations  are  made: 

1.  Continue  the  conversion  of  EME  modules  to  NBI.  The  more  modules  are 
converted  the  more  cost  effective  it  will  be.  Also,  converted  EME  modules  can 
reach  the  greatest  number  of  students  anytime  and  anywhere. 

2.  Establish  a  proven  decision  process  such  as  the  ISD  Model  to  plan,  develop,  and 
deliver  educational  programs.  This  will  help  assists  in  fulfilling  the 
requirements  for  the  successful  implementation  of  educational  and  training 
programs. 

3.  Provide  training  to  professors  who  are  willing  and  interested  in  converting  their 
EME  modules  in  the  future. 

4.  Establish  compensation  and  incentives  to  attract  and  create  interest  among 
professors  in  converting  EME  modules  to  NBI. 

D.  AREAS  FOR  FURTHER  RESEARCH 

The  conversion  of  EME  modules  to  NBI  raises  possibilities  for  future  research. 
These  include:  (1)  life  cycle  maintenance  and  upkeep  of  the  program  (2)  marketing  and 
implementation;  (3)  intellectual  property  rights;  and  (4)  empirical  testing  to  determine  the 
effects,  number  of  sites,  students,  and  different  learning  outcomes. 
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APPENDIX  FEASIBILITY  STUDY  PRODUCED  BY  ADVISOR  2.0 


BASIC  INFORMATION 


Date: 

Course  Title: 

Course  Number: 

Course  Manager: 

Operator: 

Date  of  Last  Audit: 


04-17-1998 

Budgeting  for  Defense  Health 

Module  24 

Professor  Richard  Doyle 

LT  Cesar  A.  Odvina 

04-17-1998 


REGULATIONS 


ENVIRONMENT 


Regulations 

Reasons  for  Course: 
Classification: 
Testing  Requirements: 


Combination 
Public  Domain 
Skill  /  Knowledge 


ORGANIZATIONAL 


Development  /  Revision  Time: 

Adequate 

Trainers  /  SME  Availability: 

Adequate 

Reference  Material  Availability: 

Accessible 

Consistency: 

Important 

Delivery  Time: 

Flexible 

Scheduling: 

Meets  demand 

Data  Collected: 

Large 

Course 

Objectives: 

Job  performance 

Content: 

Specific 

Status: 

New 

Trainees 

Size: 

>1000 

Location: 

Scattered 

Value  of  Trainees  Time: 

Very  high 
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TRAINING  I 


Trainers 


Delivery  Method: 
Data  Collected: 
Resistance  to  Change: 


Trainees 

Team  Work: 
Evaluation: 


Course 


Relevance: 

Content: 

Referred  To: 

Expected  Life  (years): 

Stability  (%): 

Urgency: 

Length  (hours): 


SME 

All  of  the  above 

Medium 


No 

Skill  /  Knowledge 


Prerequisite 
All  of  the  above 
Regularly 
>  5  years 
5%  to  10% 
Moderate 
4  hours 


TRAINING  II 


Content 


Hands  on  Exercises: 

No 

Personal  Safety: 

No 

Equipment  /  Data  Integrity: 

No 

Real  Equipment: 

Required,  available 

Real  Equipment  Simulation: 

Moderate 

Processor  in  Real  Equipment: 

Not  applicable 

Guided  Discussions: 

Desired 

Role  Play: 

No 

Teaming  Exercises: 

No 

Audio: 

Required 

Full  Motion  Video: 

Desired 

Still  Images: 

Required 
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INSTRUCTIONAL 


Trainees 

Receptive  to  computers: 

Yes 

Reading  Ability: 

Adequate 

Resistance  to  Change: 

Medium 

Motivation: 

Medium 

Skill  /  Proficiency: 

Diverse 

Ability  to  Travel: 

Limited 

Access  to  Computer: 

Anytime 

Access  to  Multimedia  PC: 

Anytime 

Access  to  Video  Conferencing: 

Limited 

Access  to  Internet  /  Intranet: 

Anytime 

Content 

Learning  objectives: 

Knowledge  /  Skill 

Self  Pacing: 

Most  of  course 

Share  Experiences: 

Not  required 

Apply  Knowledge: 

Complex 

SUMMARY 

Delivery  Methods  /  Media 


Scores 


Multimedia  CBT 

EPSS 

Embedded  Training 

Hypermedia 

Internet  /  Intranet 

Video  Tapes 

On-location  Seminar 

Audio  Tapes 

Video-Tele  Conferencing 

Video  Conferencing 

Classroom 

Computer  Conferencing 

Audio  Conferencing 

Simulators  /  Virtual  Reality 

Customized  CBT 

Intelligent  Tutorial 

Off-the-Shelf  CBT 


94% 

87% 

84% 

83% 

83% 

72% 

67% 

63% 

60% 

60% 

58%o 

57% 

50% 

0% 

0% 

0% 

0% 
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Hypertext  0  % 

Print  0  % 

Simple  CBT  0  % 
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